Smith family njaewikidb https://www.njae.me.uk/Main_Page MediaWiki 1.35.6 first-letter Media Special Talk User User talk Smith family Smith family talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Gadget Gadget talk Gadget definition Gadget definition talk 10 1434 1462 2007-06-12T03:47:23Z 212.71.11.150 0 wikitext text/x-wiki bgcolor="#8DA7D6" <noinclude>[[Category:Utility templates|{{PAGENAME}}]]</noinclude> 37534b6a4b895ec169104a08937b4d37474f1c63 1539 1462 2007-06-12T03:47:23Z 212.71.11.150 0 wikitext text/x-wiki bgcolor="#8DA7D6" <noinclude>[[Category:Utility templates|{{PAGENAME}}]]</noinclude> 37534b6a4b895ec169104a08937b4d37474f1c63 1463 1539 2007-08-03T09:23:09Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki bgcolor="#8DA7D6" <noinclude>[[Category:Utility templates|{{PAGENAME}}]]</noinclude> 37534b6a4b895ec169104a08937b4d37474f1c63 10 1442 1478 2007-07-05T14:26:17Z 213.56.117.250 0 wikitext text/x-wiki {| border="0" cellspacing="1" style="background-color:transparent" |- valign="middle" | {{Click|image=Wikimedia-logo.svg | link=meta:{{{1}}} | width=18px | height=18px}} | align="left" | [[meta:{{{1}}}|{{{2|MetaWiki: {{{1}}}}}}]] {{{3|}}} |}<noinclude>[[Category:Info templates|{{PAGENAME}}]]</noinclude> b211224db12655c9ccc050607003ca68e1d541a5 1555 1478 2007-07-05T14:26:17Z 213.56.117.250 0 wikitext text/x-wiki {| border="0" cellspacing="1" style="background-color:transparent" |- valign="middle" | {{Click|image=Wikimedia-logo.svg | link=meta:{{{1}}} | width=18px | height=18px}} | align="left" | [[meta:{{{1}}}|{{{2|MetaWiki: {{{1}}}}}}]] {{{3|}}} |}<noinclude>[[Category:Info templates|{{PAGENAME}}]]</noinclude> b211224db12655c9ccc050607003ca68e1d541a5 1479 1555 2007-08-03T09:23:10Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {| border="0" cellspacing="1" style="background-color:transparent" |- valign="middle" | {{Click|image=Wikimedia-logo.svg | link=meta:{{{1}}} | width=18px | height=18px}} | align="left" | [[meta:{{{1}}}|{{{2|MetaWiki: {{{1}}}}}}]] {{{3|}}} |}<noinclude>[[Category:Info templates|{{PAGENAME}}]]</noinclude> b211224db12655c9ccc050607003ca68e1d541a5 0 1 1 2007-07-16T20:42:46Z MediaWiki default 0 wikitext text/x-wiki <big>'''MediaWiki has been successfully installed.'''</big> Consult the [http://meta.wikimedia.org/wiki/Help:Contents User's Guide] for information on using the wiki software. == Getting started == * [http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list] * [http://www.mediawiki.org/wiki/Help:FAQ MediaWiki FAQ] * [http://mail.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list] 928e1deea259c70afc3513c66f29f3fcd740d8bf 1405 1 2007-07-16T20:47:23Z NJAEWikiSysop 1 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> Consult the [http://meta.wikimedia.org/wiki/Help:Contents User's Guide] for information on using the wiki software. == Getting started == * [http://www.mediawiki.org/wiki/Help:Configuration_settings Configuration settings list] * [http://www.mediawiki.org/wiki/Help:FAQ MediaWiki FAQ] * [http://mail.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list] f31ae3f25ac475be73f5f85d1567640722dcd793 1485 1405 2007-08-03T09:33:49Z NJAEWikiSysop 1 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Content will follow shortly == Jenni == Content will follow shortly == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 377f8830b8cd4696c7898eac7a27e7634c131e07 12 1405 1406 2007-07-16T20:48:38Z NJAEWikiSysop 1 wikitext text/x-wiki On a wiki, it is customary to allow readers to edit content and structure. This is a brief overview of the editing and contributing process. The editing table of contents to the right contains more information on editing functions. == Editing basics == <!--It is intended to be brief, or as an overview. Otherwise why we separate the materials into different pages? Thus it's best NOT to add new materials or expand here. Add more examples or expand in other areas like Wikitext examples. Thanks!--> ===Review policy and conventions=== :Make sure that you submit information which is relevant to the specific purpose of the wiki, or your content might be deleted. You can always use the [[Help:Talk page|talk pages]] to ask questions or check to see if your idea will be accepted. Please make note of the license your contributions will be covered with. === Start editing === :To start editing a [[w:MediaWiki|MediaWiki]] page, click the '''Edit this page''' (or just '''edit''') link at one of its edges. This brings you to the edit page: a page with a text box containing the ''[[w:wikitext|wikitext]]'' - the editable code from which the server produces the finished page. ''If you just want to experiment, please do so in the [[{{ns:4}}:Sandbox|sandbox]], not here''. ===Type your changes=== :You can just type your text. However, also using basic wiki markup (described in the next section) to make links and do simple formatting adds to the value of your contribution. [[w:Wikimedia|Wikimedia]] wikis have style guidelines available. If you follow these, your contributions will be more valuable as they won't need to be cleaned up later. ===Summarize your changes=== :Write a short [[Help:Edit summary|edit summary]] in the small field below the edit-box. You may use shorthand to describe your changes, as described in the [[w:Wikipedia:Edit summary legend|edit summary legend]]. ===Preview before saving!=== :When you have finished, click '''[[Help:Show preview|Show preview]]''' to see how your changes will look '''before''' you make them permanent. Repeat the edit/preview process until you are satisfied, then click '''Save page''' and your changes will be immediately applied to the article. Sometimes it is helpful to save in between. ==Most frequent wiki markup explained== <!--It is intended to be brief, or as an overview. Otherwise why we separate the materials into different pages? Thus it's best NOT to add new materials or expand here. Add more examples or expand in other areas like Wikitext examples. Thanks!--> Here are the most frequently used types of wiki markup. If you need more help see [[Help:Wikitext examples|Wikitext examples]]. <br clear="right" /> {| class="wikitable" ! style="width: 50%" | What it looks like ! style="width: 50%" | What you type |- | You can ''italicize text'' by putting 2 apostrophes on each side. 3 apostrophes will embolden '''the text'''. 5 apostrophes will embolden and italicize '''''the text'''''. (4 apostrophes doesn't do anything special -- there's just ''''one left over''''.) |<pre> You can ''italicize text'' by putting 2 apostrophes on each side. 3 apostrophes will embolden '''the text'''. 5 apostrophes will embolden and italicize '''''the text'''''. (4 apostrophes doesn't do anything special -- there's just ''''one left over''''.) </pre> |- | You should "sign" your comments on talk pages: <br /> - Three tildes give your user name: [[User:Example|Example]] <br /> - Four tildes give your user name plus date/time: [[User:Example|Example]] 07:46, 27 November 2005 (UTC) <br /> - Five tildes give the date/time alone: 07:46, 27 November 2005 (UTC) <br /> |<pre> You should "sign" your comments on talk pages: <br /> - Three tildes give your user name: ~~~ <br /> - Four tildes give your user name plus date/time: ~~~~ <br /> - Five tildes give the date/time alone: ~~~~~ <br /> </pre> |- | <div style="font-size:150%;border-bottom:1px solid rgb(170,170,170);">Section headings</div> ''Headings'' organize your writing into sections. The Wiki software can automatically generate a [[Help:table of contents|table of contents]] from them. <div style="font-size:132%;font-weight:bold;">Subsection</div> Using more equals signs creates a subsection. <div style="font-size:116%;font-weight:bold;">A smaller subsection</div> Don't skip levels, like from two to four equals signs. Start with 2 equals signs not 1 because 1 creates H1 tags which should be reserved for page title. |<pre> == Section headings == ''Headings'' organize your writing into sections. The Wiki software can automatically generate a table of contents from them. === Subsection === Using more equals signs creates a subsection. ==== A smaller subsection ==== Don't skip levels, like from two to four equals signs. Start with 2 equals signs not 1 because 1 creates H1 tags which should be reserved for page title. </pre> |- id="lists" | * ''Unordered [[Help:List|list]]s'' are easy to do: ** Start every line with a star. *** More stars indicate a deeper level. *: Previous item continues. ** A newline * in a list marks the end of the list. *Of course you can start again. |<pre> * ''Unordered lists'' are easy to do: ** Start every line with a star. *** More stars indicate a deeper level. *: Previous item continues. ** A new line * in a list marks the end of the list. * Of course you can start again. </pre> |- | # ''Numbered lists'' are: ## Very organized ## Easy to follow A new line marks the end of the list. # New numbering starts with 1. |<pre> # ''Numbered lists'' are: ## Very organized ## Easy to follow A new line marks the end of the list. # New numbering starts with 1. </pre> |- | : A colon (:) indents a line or paragraph. A newline starts a new paragraph. <br> Often used for discussion on [[talk pages]]. : We use 1 colon to indent once. :: We use 2 colons to indent twice. ::: 3 colons to indent 3 times, and so on. |<pre> : A colon (:) indents a line or paragraph. A newline starts a new paragraph. <br> Often used for discussion on talk pages. : We use 1 colon to indent once. :: We use 2 colons to indent twice. ::: 3 colons to indent 3 times, and so on. </pre> |- | Here's a link to the [[Main page]]. |<pre> Here's a link to the [[Main page]]. </pre> |- | [[The weather in London]] is a page that doesn't exist yet. You could create it by clicking on the link. |<pre> [[The weather in London]] is a page that doesn't exist yet. You could create it by clicking on the link. </pre> |- | You can link to a page section by its title: * [[List of cities by country#Morocco]]. If multiple sections have the same title, add a number. [[#Example section 3]] goes to the third section named "Example section". |<pre> You can link to a page section by its title: * [[List of cities by country#Morocco]]. If multiple sections have the same title, add a number. [[#Example section 3]] goes to the third section named "Example section". </pre> |} d62a5494184107dae13593ab868d7a3ef75512a9 1421 1497 2007-08-03T09:23:00Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT[[Help:Editing pages]] 8640847489ad8b1546c4043eaae6daf05ecedd3c 10 1435 1464 2007-07-28T05:51:57Z 129.83.31.2 0 wikitext text/x-wiki <div class="noprint" style="float:right; border:1px solid blue;width:200px;background-color:#fff;padding:3px;"> [[Image:Example.jpg|left|80px|Example sunflower image]] '''A little thank you...''' <br /><small>for {{{1}}}. <br />hugs, {{{2}}}</small> </div> <noinclude>[[Category:Template examples|{{PAGENAME}}]]</noinclude> 42e49911a572c3295ae1828bb8dd932b724da127 1541 1464 2007-07-28T05:51:57Z 129.83.31.2 0 wikitext text/x-wiki <div class="noprint" style="float:right; border:1px solid blue;width:200px;background-color:#fff;padding:3px;"> [[Image:Example.jpg|left|80px|Example sunflower image]] '''A little thank you...''' <br /><small>for {{{1}}}. <br />hugs, {{{2}}}</small> </div> <noinclude>[[Category:Template examples|{{PAGENAME}}]]</noinclude> 42e49911a572c3295ae1828bb8dd932b724da127 1465 1541 2007-08-03T09:23:09Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki <div class="noprint" style="float:right; border:1px solid blue;width:200px;background-color:#fff;padding:3px;"> [[Image:Example.jpg|left|80px|Example sunflower image]] '''A little thank you...''' <br /><small>for {{{1}}}. <br />hugs, {{{2}}}</small> </div> <noinclude>[[Category:Template examples|{{PAGENAME}}]]</noinclude> 42e49911a572c3295ae1828bb8dd932b724da127 12 1409 1411 1487 2007-08-03T09:22:59Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} '''Reading''' : [[Help:Navigation|Navigation]] : [[Help:Searching|Searching]] : [[Help:Tracking changes|Tracking changes]] '''Editing''' : [[Help:Editing pages|Editing pages]] : [[Help:Starting a new page|Starting a new page]] : [[Help:Formatting|Formatting]] : [[Help:Links|Links]] : [[Help:Categories|Categories]] : Advanced editing: :: [[Help:Images|Images]] :: [[Help:Tables|Tables]] :: [[Help:Templates|Templates]] :: [[Help:Variables|Variables]] :: [[Help:Managing files|Managing files]] : Personal customizing: :: [[Help:Preferences|Preferences]] :: [[Help:Skins|Skins]] : Wiki administrator tools: :: [[Help:Range blocks|Range IP blocks]] '''Server administration''' : [[Help:Managing user rights|Managing user rights]] : [[Help:Namespaces|Namespaces]] : [[Help:Copying|Copying 'Help']] ==See also== [http://meta.wikimedia.org/wiki/Help:Displaying_a_formula Mathematical formula] [[Category:Help| ]] 764bb7e4a47850777b11ff22c0cecdf1f26457d3 12 1410 1413 1489 2007-08-03T09:22:59Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} If you look at a page in MediaWiki, you'll find three main navigation elements: The sidebar on the left gives you access to important pages in the Wiki like recent changes or file upload. At the top of the page are the links which belong to the page currently displayed: its associated discussion page, the version history, and - most notably - the edit link. In the top right corner you'll find, as an anonymous user, the link to create an account and login (they are on the same page). As a logged-in user you have a collection of personal links, like the one to your user page or your preferences. == Sidebar == [[Image:M-en-sidebar.png|framed|right|Example sidebar, shown on the left of the page]] The sidebar is displayed on the left edge of the page below the site logo (if using the default MonoBook skin). This sidebar gives you access to important pages in the Wiki like recent changes or file upload. === Navigation === A click on the logo brings you back to the start page of the wiki. The links take you to important pages in the wiki, they can be configured by site administrators of the individual wikis {{Admin tip|tip=You can customize the links in the navigation section via [[Special:Allmessages]]. For example edit [[MediaWiki:Sitesupport]] (displayed name) and [[MediaWiki:Sitesupport-url]] (target page). Or create a menu on your own in [[MediaWiki:Sidebar]].}} === Toolbox === The toolbox contains a selection of links which change depending on what type of page you are viewing. On all pages (except special pages): * ''What links here'' takes you to a special page that lists the pages on this wiki which contain a link to the current page. This is helpful when you are looking for pages of related information. The ''What links here'' information can also be useful when you are refactoring wiki pages and need to check whether links to this page are still relevant after changes in the current page. * The ''Related changes'' tool lists all recent changes in the pages linked to from the current page. Recent changes to all relevant template pages are included in the resulting page list. The "Hide minor edits" option that can be set in the user [[Help:Preferences|preferences]] applies, among other things, to ''Related Changes''. On all pages (including special pages): * ''Upload file'' displays a special page that allows logged-in users to upload images and other files to the wiki. Uploaded files can be linked-from or embedded-in wiki pages. Uploading files, viewing files on the server, including them in wiki pages and managing the uploaded files is discussed in the [[Help:Managing files|managing files]] section of this manual. This is not displayed if file uploading has been disabled or not enabled in the first place. {{Admin tip|tip=To enable file uploading someone with access to the MediaWiki installation files needs to edit the <code>LocalSettings.php</code> file and uncomment or add the option <code>$wgEnableUploads = true;</code>. Uploaded files will be stored in the images folder specified by the <code>$wgUploadPath</code> variable in the <code>LocalSettings.php</code>. This directory must be writable if file uploads is enabled. The [http://www.mediawiki.org/wiki/Manual:LocalSettings.php#Upload_location upload location] and [http://www.mediawiki.org/wiki/Manual:LocalSettings.php#Image_uploads image uploads]] settings are described in more detail on the MetaWiki [http://www.mediawiki.org/wiki/Manual:LocalSettings.php LocalSettings.php] page.}} * The ''Special pages'' tool lists the MediaWiki special pages. In MediaWiki terminology, a special page is one that presents information about the Wiki and/or allows access to administration activities for the wiki. For example, a list of users registered with the wiki, statistics about the wiki such as the number of pages and number of page edits, system logs, a list of orphaned pages, and so on. These special pages are commonly generated when the special page is loaded rather than being stored in the wiki database. :''The function and use of the default special pages can be found in the [[Help:Special pages|special pages]] section of this manual.'' == Page Tabs == [[Image:M-en-pagetabs.png|framed|right|Default page tabs at the top of the page]] The page tabs are displayed at the top of the article to the right of the site logo (if using the default MonoBook skin). These tabs allow you to perform actions or view pages that are related to the current article. The available default actions include: viewing, editing, and discussing the current article. The specific tabs displayed on your pages depend on whether or not you are logged into the wiki and whether you have sysop (administrator) privileges on the wiki. On special pages only the namespace tab is displayed. ;Default for all users : ''[[Help:Namespaces|namespace]]'' (article, help, special page, template, user page etc.) : ''discussion'' : ''edit'' (may be view source if anonymous editing is enabled, the page is in the MediaWiki namespace or the page is protected) : ''history'' ;Extra tabs for logged in users: : ''move'' : ''watch'' ;Extra tabs for sysops: : ''protect'' : ''delete'' Extra tabs may be added, or some taken away, by wiki administrators using JavaScript or adding extensions so these may be different depending on which wiki you are using. == User Links == [[Image:M-en-userlinks.png|framed|right|Default user links at the top right of the page]] The user links are displayed at the top far right of the article (if using the default MonoBook skin). These tabs allow the logged-in user to view and edit their user page and wiki preferences. Additionally, the user links allow the user to quickly access their contributions to the wiki and logout. For anonymous users the user links is replaced by a link to the wiki login page or, if enabled, a link to your ip address and your ip address's talk page. * ''<username>'' *: This links to your user page which is where you can put information about yourself, store bits of information you want to remember or whatever else you fancy. * ''my talk'' *: This links to your discussion page, where people can leave messages for you. * ''preferences'' *: Allows you to change your personal site preferences. * ''my watchlist'' *: A list of all pages that you are watching. Pages can be added to this list by clicking 'watch' at the top of the page. * ''my contributions'' *: A list of all contributions you have made to the wiki. * ''log out'' *: Click this link to log out of the wiki. [[Category:Help|Navigation]] dac0d834e7134c30b4774792355b47a28eaaacfa 12 1411 1415 1491 2007-08-03T09:22:59Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} The quickest way to find information in MediaWiki is to look it up directly. On the left-hand side of your screen there is a '''Search''' box with two buttons under it labeled "Go" and "Search". <div style="border:1px solid #ababff; background: #fdfdff; padding: 1em; margin: 1.1em 1em 1em 0em; font-size:105%"> Put your keyword in the searchbox. * <font face="MS Sans Serif,Arial">'''Go'''</font> - (or ''Enter'' on keyboard) will take you automatically to the article, if it exists. * <font face="MS Sans Serif,Arial">'''Search'''</font> - will search the text of all pages on the wiki (with some restrictions, see below). </div> If you clicked 'search', or you clicked 'go' and the page you were looking for did not exist, you will be presented with a list of articles that matched your search criteria (or a message that no matches were found). == How it works == Here's how the search works: * Only the article content is searched - the page title is ignored. * The article content is searched in its raw (wikitext) form - i.e. it searches the text that appears in the edit box when you click 'edit', not the rendered page. This means that content coming from an included template will not be picked up, but the target of piped links will be. * Even if you enclose a phrase in quotes, the search looks for each word individually. e.g. if you enter "world war 2" it will return pages that contain "world" and "war" and "2". * The search is not case-sensitive, so "MediaWiki", "mediawiki" and "MEDIAWIKI" all give the same result. == Restricting the search == If you click the 'Search' button without filling in anything, you will be taken to 'Special:Search' which gives you extra searching options (also available from any search results list) You may find it useful to restrict a search to pages within a particular [[Help:Namespaces|namespace]] e.g. only search within the 'Talk' pages. Tick the namespaces you require for this search. By default only the [[Help:Namespaces|namespaces]] specified in your [[Help:Preferences|preferences]] will be searched. Logged-in users can change their preferences to specify the namespaces they want to search by default. __NOTOC__ [[Category:Help|Searching]] ac95b57af14452387e5cce87b4e1d3331328d642 12 1412 1417 1493 2007-08-03T09:23:00Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} [[Image:M-en-recentchanges.png|thumb|370px|'''Recent changes''' shows the latest changes to articles, file uploads, deletions and page moves]] MediaWiki offers a collection of special pages and tools to keep track of what is going on in the wiki. So you can watch for example: * recent changes to all pages * newly created pages * the contributions of one specific user * the revision history: all changes made to one page The most interesting special page is '''[[Special:Recentchanges]]'''. It displays all edits, file uploads, page moves, deletions and other actions done in the wiki. In the menu on top it offers a collection of links to customize your display: limit the number of changes shown, the number of days or restrict it to edits to a certain namespace. You can also hide edits marked as minor (don't forget that major changes can be flagged by a user as minor anyway). One line in recentchanges consists of several links: * ''diff'' displays the difference to the previous revision of the page, * ''hist'' links to the revision history of the page, * the link with the full title of the page brings you to the current version. If the title is in bold, it indicates that it is on your watchlist. Next is a flag describing the article modification type: *''N'' signalizes a new page, *''m'' a minor edit, *''b'' an edit made by a bot. Following the timestamp the user is mentioned with a link to his user and talk page and to his contributions. In italic follows a comment if the user has submitted one. === See also === * [[Special:Newpages]] * Related changes: [[Special:Recentchangeslinked/Pagename]] * User contributions: [[Special:Contributions/User]] * History * [[Special:Watchlist]] {{Admin tip|tip=You may want to add important pages in your wiki to [[MediaWiki:Recentchangestext]], the intro shown at [[Special:Recentchanges]]}} [[Category:Help|Tracking changes]] 2ea46d15eb1f30c7150a890d2edb9ba3a8dbe041 12 1413 1419 1495 2007-08-03T09:23:00Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Easy editing of pages is what wikis are made for. It only takes a few clicks. == Editing new pages == If the page you wish to edit does not exist yet, see [[Help:Starting a new page]] == Editing existing pages == * Click the '''edit''' [[Help:Navigation#Page Tabs|page tab]] * Make changes to the text in the edit box. Make sure you write clearly and concisely, and make sure your edit does something to improve the page. If you're making normal changes to the text like fixing spelling mistakes or grammar, inserting new sentences, etc, then you don't have to worry too much about formatting. When you do need to use some type of formatting, you do it using wiki syntax, see [[Help:Formatting]] for some of the common types of formatting used. * Optionally, enter a short note in the '''Summary''' box describing your changes. * Optionally preview your changes with the '''Preview''' button. * Click the '''Save page''' button. * One can rename pages as well See [http://meta.wikimedia.org/wiki/Help:Moving_a_page Help:Moving_a_page] === Discussion === Every article has its own discussion page where you can ask questions, make suggestions, or discuss corrections. Click the '''discussion''' [[Help:Navigation#Page Tabs|page tab]] to reach it. You can sign your message by writing 4 tildes (<nowiki>~~~~</nowiki>). The wiki software turns them into the current time and your username or your current IP address if you are not logged in. == External Links == [http://meta.wikimedia.org/wiki/Help:Editing Help:Editing] [[Category:Help|Editing pages]] 80611681e1843d02ef23a0e6a8b88f77f9f09660 12 1414 1423 1422 2007-08-03T09:23:00Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} There are several ways to start a new page. == Using Wikilinks == MediaWiki makes it very easy to link wiki pages using a standard syntax (see [[Help:Links|Links]]). If you (or anyone else) creates a link to an article that doesn't exist yet, the link will be coloured red, <span style="color: #ba0000">like this</span>. Clicking a red link, will take you to the edit page for the new article. Simply type your text, click save and the new page will be created. Once the page has been created, the link will change from <span style="color: #ba0000">red</span> to <span style="color: #002bb8;">blue</span> (<span style="color: #5a3696;">purple</span> for pages you've visited) indicating that the article now exists. == Using the URL == You can use the wiki's URL for creating a new page. The URL to an article of the wiki is usually something like this: *<code><nowiki>http://www.example.net/index.php/</nowiki>'''ARTICLE'''</code> &nbsp;&nbsp; or *<code><nowiki>http://www.example.net/wiki/</nowiki>'''ARTICLE'''</code> If you replace <code>'''ARTICLE'''</code> with the name of the page you wish to create, you will be taken to a blank page which indicates that no article of that name exists yet. Clicking the "''edit''" [[Help:Navigation#Page Tabs|page tab]] at the top of the page will take you to the edit page for that article, where you can create the new page by typing your text, and clicking submit. == From the search page == If you search for a page that doesn't exist (using the search box and 'go' button on the left of the page) then you will be provided with a link to create the new page. (Note that this technique doesn't work if you use the 'search' button). == Using the Inputbox extension == {{Admin tip|tip=You need to download and install the [[meta:Inputbox|"inputbox extension"]] for this method to work.}} If the Inputbox extension is available on the wiki you can start a new page by typing in your article's name and clicking the "create article" button, as in the example below. {| align="center" style="border:1px solid black" width="60%" | <inputbox> type=create width=45 </inputbox> |} == Create redirects to your new page == Don't forget to setup redirects when you create a page. If you think another person may search for the page you've created by using a different name or spelling, please create the proper redirect(s). To create a redirect, the '''first line''' of the redirect page should read <code><nowiki>#REDIRECT [[Destination]]</nowiki></code>, where <code>Destination</code> is the page to which people should be redirected. This must appear as the very first line of the page. {{Admin tip|tip= If desired, once you create your new page, you should protect the page and allow only the type of users you would like to edit the page. }} [[Category:Help|Starting a new page]] 5f6a44fa722cdd8b8939018e1d3b869002845098 12 1415 1425 1501 2007-08-03T09:23:01Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} You can format your text using wiki markup. This consists of normal characters like asterisks, single quotes or equation marks which have a special function in the wiki, sometimes depending on their position. For example, to format a word in ''italic'', you include it in two single quotes like <tt><nowiki>''this''</nowiki></tt> == Text formatting markup == {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Description''' |{{Hl2}} |'''You type''' |{{Hl2}} |'''You get''' |- | colspan="3" align="center" | ''applies anywhere'' |- |Italic text |<tt><nowiki>''italic''</nowiki></tt> |''italic'' |- | Bold text |<tt><nowiki>'''bold'''</nowiki></tt> |'''bold''' |- | Bold and italic |<tt><nowiki>'''''bold & italic'''''</nowiki></tt> |'''''bold & italic''''' |- |Escape wiki markup |<tt><nowiki><nowiki>no ''markup''</nowiki></nowiki></tt> |<nowiki>no ''markup''</nowiki> |- | colspan="3" align="center" | ''only at the beginning of the line'' |- |Headings of<br /> different sizes | <tt><nowiki>==level 1==</nowiki></tt><br /> <tt><nowiki>===level 2===</nowiki></tt><br /> <tt><nowiki>====level 3====</nowiki></tt><br /> <tt><nowiki>=====level 4=====</nowiki></tt> | ==Level 1== ===Level 2=== ====Level 3==== =====Level 4===== |- |Horizontal rule | <tt><nowiki>----</nowiki></tt> | ---- |- |Bullet list | &#42; one<br> &#42; two<br> &#42; three<br> &#42;&#42; three and one-third<br> &#42;&#42; three and two-thirds<br> | * one * two * three ** three and one-third ** three and two-thirds |- |Numbered list | &#35; one<br /> &#35; two<nowiki><br></nowiki>spanning several lines<nowiki><br></nowiki>without breaking the numbering<br /> &#35; three<br /> &#35;&#35; three point one<br /> &#35;&#35; three point two<br /> | # one # two<br>spanning several lines<br>without breaking the numbering # three ## three point one ## three point two |- | Mixture of bulleted<br /> and numbered lists | &#35; one<br /> &#35; two<br /> &#35;&#42; two point one<br /> &#35;&#42; two point two<br /> | # one # two #* two point one #* two point two |- |Definition list | &#59;Definition<br /> &#58;item 1<br /> &#58;item 2<br /> | ;Definition :item 1 :item 2 |- |Preformatted text | &nbsp;&nbsp;preformatted text is done with<br /> &nbsp;&nbsp;a space at the<br /> &nbsp;&nbsp;beginning of the line<br /> | preformatted text is done with a space at the beginning of the line |} == Paragraphs == MediaWiki ignores normal line breaks. To start a new paragraph, leave an empty line. You can also start a new line with the HTML tag <tt><nowiki><br&nbsp;/></nowiki></tt>. == HTML == Some HTML-Tags are allowed in MediaWiki, for example <tt><nowiki><code></nowiki></tt>, <tt><nowiki><div></nowiki></tt>, <tt><nowiki><span></nowiki></tt> and <tt><nowiki><font></nowiki></tt>. {{Admin tip|tip=If you trust your users you can allow full HTML by setting <code>$wgRawHtml = true;</code> in LocalSettings.php.}} == Other formatting == Beyond the text formatting markup shown above, here are some other formatting references: *[[Help:Links|Links]] *[[Help:Images|Images]] *[[Help:Tables|Tables]] [http://en.wikipedia.org/wiki/Wikipedia:WikiProject_User_Page_Help/Do-It-Yourself/Formatting_Text#Underlining More Formatting from Wikipedia] == External Links == [http://meta.wikimedia.org/wiki/Help:Editing Help:Editing] __NOTOC__ __NOEDITSECTION__ [[Category:Help|Formatting]] ed8bc65ec07575e2da066adadb871a7e0b367dec 12 1416 1427 1426 2007-08-03T09:23:01Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} There are four sorts of links in MediaWiki: # internal links to other pages in the wiki # external links to websites # inter-wiki links (links to other wikis) # inter-language links to other language versions of the same wiki To add an internal link, enclose the name of the page you want to link to in double square brackets. When you save the page, you'll see the new link pointing to your page. If the page exists already, it is displayed in blue, empty pages are displayed in red. Selflinks to the current page are not transformed in URLs but displayed in bold. The first letter of the target page is automatically capitalized and spaces are represented as underscores (typing an underscore in the link will have a similar effect as typing a space, but is not recommended, since the underscore will also be shown in the text). == How to link == {| {{Prettytable}} |- |{{Hl3}}|'''Description''' |{{Hl3}}|'''You type''' |{{Hl3}}|'''You get''' |- |Internal link |<tt><nowiki>[[Main Page]]</nowiki></tt> |[[Main Page]] |- |Category link |<tt><nowiki>[[:Category:Help]]</nowiki></tt> |[[:Category:Help]] |- |Piped link |<tt><nowiki>[[Main Page|different text]]</nowiki></tt> |[[Main Page|different text]] |- |Anchor link |<tt><nowiki>[[#External links|Anchor link]]</nowiki></tt> |[[#External links|Anchor link]] |- |External link |<tt><nowiki>http://mediawiki.org</nowiki></tt> |http://mediawiki.org |- |External link from internal image |<tt><nowiki>[http://targetURL {{Server}}/wiki/images/imagelocation</nowiki></tt> |Internal image |- |External link,<br /> different title |<tt><nowiki>[http://mediawiki.org MediaWiki]</nowiki></tt> |[http://mediawiki.org MediaWiki] |- |External link,<br /> unnamed |<tt><nowiki>[http://mediawiki.org]</nowiki></tt> |[http://mediawiki.org] |- |External link,<br /> same host unnamed |<tt><nowiki>[http://{{SERVERNAME}}/pagename]</nowiki></tt> |[http://{{SERVERNAME}}/pagename] |- |Interwiki link |<tt><nowiki>[[Wikipedia:MediaWiki]]</nowiki></tt> |[[Wikipedia:MediaWiki]] |- |mailto |<tt><nowiki>mailto:info@example.org</nowiki></tt> |mailto:info@example.org |- |mailto unnamed |<tt><nowiki>[mailto:info@example.org]</nowiki></tt> |[mailto:info@example.org] |- |mailto named |<tt><nowiki>[mailto:info@example.org info]</nowiki></tt> |[mailto:info@example.org info] |- |redirect |<tt><nowiki>#REDIRECT [[Main Page]]</nowiki></tt> | &rarr; [[Main Page]] |} == External links == [http://meta.wikimedia.org/wiki/Help:Link Help:Link] [[Category:Help|Links]] 8c80f302b017390417bf030ab2a58a8a3a540157 12 1417 1429 1505 2007-08-03T09:23:02Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} To add an article to a category put the following at the end of the page you are editing... <nowiki>[[Category:{Name}]]</nowiki> where {Name} is the name of the category you want to add it to. Any number of category tags may be added to the page - the page will be listed in all of them. You can also specify an additional {Sort} parameter that dictates where the page will appear, alphabetically, within the category. This is achieved by using the following markup: <nowiki>[[Category:{Name}|{Sort}]]</nowiki> So for example, to add this page to the 'Help' category, you would use: <nowiki>[[Category:Help|Categories]]</nowiki> Note that we used 'Categories' as the sort parameter. Without this the page would be listed under 'H' for 'Help:Categories', instead of under 'C', which is more useful. Other situations where you might want to use the sort parameter is when you have articles about people that are titled as <code>FirstName LastName</code> but within the category you want them listed as <code>LastName, FirstName</code>. Another way to sort the article in the correct letter without the namespace is <nowiki>[[Category:Help|{{PAGENAME}}]]</nowiki> This is extremely helpful when using templates which include a category tag. ''Note: the {sort} parameter does '''not''' affect how the page title is displayed within the category listing, just how it is ordered. In the above example, the link to this page will still be 'Help:Categories', and not 'Categories' as you might expect!'' == Linking to Category Pages == To create a link to a category page: <nowiki>[[:Category:{name}]]</nowiki> If you were linking to the Category Page for Help on MediaWiki, the link would look like this: [[:Category:Help]] If you want to display alternate text for the link: <nowiki>[[:Category:{name}|{alternate text}]]</nowiki> Here is an example of the same link to the Category Page for Help on MediaWiki as above, but with alternative text: [[:Category:Help|MediaWiki Help Index]] == Categorize Categories== Categories themselves and other uploaded files like Pictures can be categorized exactly like normal pages. It is useful to connect the article-categories with categories already in place to establish connections and hierarchies. To this end, after saving the article, follow the category links at the end of the page to see, if the category is already in place and if not, categorize them until you connect them with an existing category. ==See also== * [http://meta.wikimedia.org/wiki/Help:Category WikiMedia:Help:Category] [[Category:Help|Categories]] [[Category:Category| ]] 58bc26cf65328aae77774d4aea568a8a4f83afdc 12 1418 1431 1507 2007-08-03T09:23:03Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} __TOC__ {| {{Prettytable}} |- |{{Hl3}}|'''Description''' |{{Hl3}}|'''You type''' |{{Hl3}}|'''You get''' |- |Embed image<br /> (with alt text) | <code><nowiki>[[Image:Example.jpg|Sunflowers]]</nowiki></code> | [[Image:Example.jpg|Sunflowers]] |- |Link to description page | <code><nowiki>[[:Image:Example.jpg]]</nowiki></code><br /> <code><nowiki>[[:Image:Example.jpg|Sunflowers]]</nowiki></code> | [[:Image:Example.jpg]]<br /> [[:Image:Example.jpg|Sunflowers]] |- |Link directly to file | <code><nowiki>[[Media:Example.jpg]]</nowiki></code><br /> <code><nowiki>[[Media:Example.jpg|Sunflowers]]</nowiki></code> | [[Media:Example.jpg]]<br /> [[Media:Example.jpg|Sunflowers]] |- |Thumbnail<br /> (centered, 100 pixels<br /> wide, with caption) | <code><nowiki>[[Image:Example.jpg|center|thumb|100px|Sunflowers]]</nowiki></code> | [[Image:Example.jpg|center|thumb|100px|Sunflowers]] |- |Border<br /> (100 pixels) <br /> Results in a very small gray border <!-- Note: because the fine gray border can not be seen when used on the "Sunflowers"-image an image is used with a withe background --> | <code><nowiki>[[Image:Exampe.jpg|border|100px]]</nowiki></code> | [[Image:Example.jpg|border|100px]]<br> (note that the border is not visible when used with this image) |- |Frameless<br />Like thumbnail, respect user preferences for image width but without border and no right float. | <code><nowiki>[[Image:Example.jpg|frameless]]</nowiki></code> | [[Image:Example.jpg|frameless]] |} ==Uploaded files== To use an image or other file which has been [[Help:Managing files|uploaded to the wiki]], use: *<code><nowiki>[[Image:Example.jpg]]</nowiki></code> If you add a pipe (|) and some text after the filename, the text will be used as alternative text for text-only browsers: *<code><nowiki>[[Image:Example.jpg|alternative text]]</nowiki></code> If you don't want to display the image, you can link to the file's description page, by adding a colon: *<code><nowiki>[[:Image:Example.jpg]]</nowiki></code> To bypass the description page and link directly to a file , use the "Media" pseudo-namespace: *<code><nowiki>[[Media:Example.jpg]]</nowiki></code> You can also make piped links if you want some other text to be used: *<code><nowiki>[[:Image:Example.jpg|link text]]</nowiki></code> *<code><nowiki>[[Media:Example.jpg|link text]]</nowiki></code> ===Advanced options=== The full syntax for displaying an image is: <code><nowiki>[[Image:{name}|{options}]]</nowiki></code> Where options can be zero or more of the following, separated by pipes: *<code>thumb</code>, <code>thumbnail</code>, or <code>frame</code>: Controls how the image is formatted *<code>left</code>, <code>right</code>, <code>center</code>, <code>none</code>: Controls the alignment of the image on the page *<code>{width}px</code>: Resizes the image to the given width in pixels *<code>{caption text}</code> The options can be given in any order. If a given option does not match any of the other possibilities, it is assumed to be the caption text. Caption text can contain wiki links or other formatting. ===Gallery of images=== It's easy to make a gallery of thumbnails with the <code><nowiki><gallery></nowiki></code> tag. The syntax is: <pre> <gallery> Image:{filename}|{caption} Image:{filename}|{caption} {...} </gallery> </pre> Captions are optional, and may contain wiki links or other formatting. for example: <pre> <gallery> Image:Example.jpg|Item 1 Image:Example.jpg|a link to [[Help:Contents]] Image:Example.jpg Image:Example.jpg Image:Example.jpg|''italic caption'' </gallery> </pre> is formatted as: <gallery> Image:Example.jpg|Item 1 Image:Example.jpg|a link to [[Help:Contents]] Image:Example.jpg Image:Example.jpg Image:Example.jpg|''italic caption'' </gallery> ==Other files== You can link to an external file using the same syntax used for linking to an external web page. *<code><nowiki>[http://url.for/some/image.png]</nowiki></code> Or with different text: *<code><nowiki>[http://url.for/some/image.png link text here]</nowiki></code> If it is enabled on your wiki (see [http://www.mediawiki.org/wiki/Manual:$wgAllowExternalImages Manual:$wgAllowExternalImages]), you can also embed external images. To do that, simply insert the image's url: *<code><nowiki>http://url.for/some/image.png</nowiki></code> == Administration of images/files == See here: [http://www.mediawiki.org/wiki/Manual:Image_Administration Manual:Image Administration] ==External Links== [http://meta.wikimedia.org/wiki/Help:Images_and_other_uploaded_files Images and other uploaded files] [[Category:Help|Images]] cf25ea60b9a82f25443430313cc49ec8df4c4c1b 12 1419 1433 1432 2007-08-03T09:23:03Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} If you have standard texts you want to include on several pages, the MediaWiki template feature comes into play (like the tag above which is included in the pages of the Help namespace). ==Creating a template== Template names are prefixed with <code>Template:</code>, you create them like any other wiki page. ==Using a template== Templates are wiki pages which can be used in other pages in three ways: *<code><nowiki>{{Name}}</nowiki></code> includes the content of the template at the moment the page containing it is fetched <nowiki>"[[Template:Name]]"</nowiki>. *<code><nowiki>{{subst:Name}}</nowiki></code> inserts the content of the template into the code of the page in a form that is editable normally <nowiki>"[[Template:Name]]"</nowiki> after you have saved your text. *<code><nowiki>{{msgnw:Name}}</nowiki></code> when the page containing it is fetched, includes the template in a form that displays it as raw wiki syntax, like <code><nowiki>&lt;nowiki&gt;</nowiki></code> does ==Using parameters in templates== <div style="float:right"> {| {{Prettytable}} |- |{{Hl2}} colspan="2" align="center" |'''Template with numbered parameters''' |- | colspan="2" | <pre><nowiki> '''A little thank you...'''<br> <small>for {{{1}}}.<br> hugs, {{{2}}}</small> </nowiki></pre> |- |{{Hl2}}|'''You type''' |{{Hl2}}|'''You get''' |- |<code><nowiki>{{Thankyou|all|Joe}}</nowiki></code> | {{Thankyou|all|Joe}} |- |{{Hl2}} colspan="2" align="center" |'''with named parameters''' |- | colspan="2" | <pre><nowiki> '''A little thank you...'''<br> <small>for {{{reason}}}.<br> hugs, {{{signature}}}</small> </nowiki></pre> |- |{{Hl2}}|'''You type''' |{{Hl2}}|'''You get''' |- |<pre><nowiki>{{Thankyou |reason=all |signature=Joe}}</nowiki></pre> | {{Thankyou|all|Joe}} |} </div> You can define parameters in templates either numbered as <code><nowiki>{{{1}}}</nowiki></code> or named <code><nowiki>{{{param}}}</nowiki></code>. '''Example:''' You want a little thank you note you can put on the talk page of other users. It will contain a reason and your signature. You could create [[Template:Thankyou]] to enter your text, as in the example in the table. When using the template on a page, you fill in the parameter values, separated by a pipe char (|): <code><nowiki>{{Thankyou|all|Joe}}</nowiki></code>. For named parameters use "name=value" pairs separated by a pipe char: <code><nowiki>{{Thankyou|reason=all|signature=Joe}}</nowiki></code>. The advantage of using named parameters in your template is that they are flexible in order. It also makes the template easier to understand if you have many parameters. If you want to change the order of numbered parameters, you have to mention them explicitly: <code><nowiki>{{Thankyou|2=Joe|1=all}}</nowiki></code>. You can also provide default values for parameters, i.e. values that are going to be used if no value is provided for a parameter. For example, <code><nowiki>{{{reason|all}}}</nowiki></code> would result in ''"all"'' if no value was provided for the parameter <tt>reason</tt>. ==Control template inclusion== You can control template inclusion by the use of <code><nowiki><noinclude></nowiki></code> and <code><nowiki><includeonly></nowiki></code> tags. Anything between <code><nowiki><noinclude></nowiki></code> and <code><nowiki></noinclude></nowiki></code> will be processed and displayed only when the page is being viewed directly, not included. Possible applications are: * Categorising templates * Interlanguage links to similar templates in other languages * Explanatory text about how to use the template The converse is <code><nowiki><includeonly></nowiki></code>. Text between <code><nowiki><includeonly></nowiki></code> and <code><nowiki></includeonly></nowiki></code> will be processed and displayed only when the page is being included. The obvious application is to add all pages containing a given template to a category. Note that the usual update problems apply -- if you change the categories inside a template, the categories of the referring pages won't be updated until those pages are edited. ==Organizing templates== For templates to be effective users need to find them and be able to use them. A simple technique is to include an example on the template page. For example: <div style="display:table; width:auto;"><pre> <noinclude> ==See Also== *[[Help:Variables]] -- fancy stuff you may find in some templates ==See Also== *[[Help:External searches]] -- a template special use case example *[[Help:Variables]] -- fancy stuff you may find in some templates ==External links== [http://meta.wikimedia.org/wiki/Help:Template Help:Template] [[Category:Help|{{PAGENAME}}]] 4075b68f88ee16d2514f0a0f8ea4274ee868da44 1486 1509 2007-08-03T15:21:09Z NJAEWikiSysop 1 wikitext text/x-wiki {{PD Help Page}} If you have standard texts you want to include on several pages, the MediaWiki template feature comes into play (like the tag above which is included in the pages of the Help namespace). ==Creating a template== Template names are prefixed with <code>Template:</code>, you create them like any other wiki page. ==Using a template== Templates are wiki pages which can be used in other pages in three ways: *<code><nowiki>{{Name}}</nowiki></code> includes the content of the template at the moment the page containing it is fetched <nowiki>"[[Template:Name]]"</nowiki>. *<code><nowiki>{{subst:Name}}</nowiki></code> inserts the content of the template into the code of the page in a form that is editable normally <nowiki>"[[Template:Name]]"</nowiki> after you have saved your text. *<code><nowiki>{{msgnw:Name}}</nowiki></code> when the page containing it is fetched, includes the template in a form that displays it as raw wiki syntax, like <code><nowiki>&lt;nowiki&gt;</nowiki></code> does ==Using parameters in templates== <div style="float:right"> {| {{Prettytable}} |- |{{Hl2}} colspan="2" align="center" |'''Template with numbered parameters''' |- | colspan="2" | <pre><nowiki> '''A little thank you...'''<br> <small>for {{{1}}}.<br> hugs, {{{2}}}</small> </nowiki></pre> |- |{{Hl2}}|'''You type''' |{{Hl2}}|'''You get''' |- |<code><nowiki>{{Thankyou|all|Joe}}</nowiki></code> | {{Thankyou|all|Joe}} |- |{{Hl2}} colspan="2" align="center" |'''with named parameters''' |- | colspan="2" | <pre><nowiki> '''A little thank you...'''<br> <small>for {{{reason}}}.<br> hugs, {{{signature}}}</small> </nowiki></pre> |- |{{Hl2}}|'''You type''' |{{Hl2}}|'''You get''' |- |<pre><nowiki>{{Thankyou |reason=all |signature=Joe}}</nowiki></pre> | {{Thankyou|all|Joe}} |} </div> You can define parameters in templates either numbered as <code><nowiki>{{{1}}}</nowiki></code> or named <code><nowiki>{{{param}}}</nowiki></code>. '''Example:''' You want a little thank you note you can put on the talk page of other users. It will contain a reason and your signature. You could create [[Template:Thankyou]] to enter your text, as in the example in the table. When using the template on a page, you fill in the parameter values, separated by a pipe char (|): <code><nowiki>{{Thankyou|all|Joe}}</nowiki></code>. For named parameters use "name=value" pairs separated by a pipe char: <code><nowiki>{{Thankyou|reason=all|signature=Joe}}</nowiki></code>. The advantage of using named parameters in your template is that they are flexible in order. It also makes the template easier to understand if you have many parameters. If you want to change the order of numbered parameters, you have to mention them explicitly: <code><nowiki>{{Thankyou|2=Joe|1=all}}</nowiki></code>. You can also provide default values for parameters, i.e. values that are going to be used if no value is provided for a parameter. For example, <code><nowiki>{{{reason|all}}}</nowiki></code> would result in ''"all"'' if no value was provided for the parameter <tt>reason</tt>. ==Control template inclusion== You can control template inclusion by the use of <code><nowiki><noinclude></nowiki></code> and <code><nowiki><includeonly></nowiki></code> tags. Anything between <code><nowiki><noinclude></nowiki></code> and <code><nowiki></noinclude></nowiki></code> will be processed and displayed only when the page is being viewed directly, not included. Possible applications are: * Categorising templates * Interlanguage links to similar templates in other languages * Explanatory text about how to use the template The converse is <code><nowiki><includeonly></nowiki></code>. Text between <code><nowiki><includeonly></nowiki></code> and <code><nowiki></includeonly></nowiki></code> will be processed and displayed only when the page is being included. The obvious application is to add all pages containing a given template to a category. Note that the usual update problems apply -- if you change the categories inside a template, the categories of the referring pages won't be updated until those pages are edited. ==Organizing templates== For templates to be effective users need to find them and be able to use them. A simple technique is to include an example on the template page. For example: <div style="display:table; width:auto;"><pre> <noinclude> ==Usage== Allows to establish a link to a subject: {{NameOfTemplate|Term1+Term2+Term3}} </noinclude> </pre></div> Then, an editor can simply copy and paste the example to create a similar page. ==See Also== *[[Help:Variables]] -- fancy stuff you may find in some templates ==External links== [http://meta.wikimedia.org/wiki/Help:Template Help:Template] [[Category:Help|{{PAGENAME}}]] ad8e8a7fa096c0ae3a2dece6779d12daf7029617 12 1420 1435 1511 2007-08-03T09:23:04Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Tables may be authored in wiki pages using either HTML table elements directly, or using wikicode formatting to define the table. HTML table elements and their use are well described on various web pages and will not be discussed here. The benefit of wikicode is that the table is constructed of character symbols which tend to make it easier to perceive the table structure in the article editing view compared to HTML table elements. A good general rule of thumb is to avoid using a table unless you need to. Table markup often complicates page editing. == Wiki table markup summary == {|cellpadding="5" cellspacing="0" border="1" width="600" |<nowiki>{|</nowiki> || start '''table''' |- |<nowiki>|+</nowiki> || table '''caption,''' ''optional;'' only one per table and between table start and first row |- |<nowiki>|-</nowiki> || '''table row,''' ''optional on first row'' -- wiki engine assumes the first row |- |<nowiki>!</nowiki> || '''table header''' cell, ''optional.'' Consecutive table headers may be added on same line separated by double marks (!!) or start on new lines, each with its own single mark (!). |- |<nowiki>|</nowiki> || '''table data''' cell, ''required!'' Consecutive table data cells may be added on same line separated by double marks (<nowiki>||</nowiki>) or start on new lines, each with its own single mark (<nowiki>|</nowiki>). |- |<nowiki>|}</nowiki> || end '''table''' |} *The above marks must '''start on a new line''' except the double || and !! for optionally adding consecutive cells to a line. *'''XHTML attributes.''' Each mark, except table end, optionally accepts one or more XHTML attributes. Attributes must be on the same line as the mark. Separate attributes from each other with a single space. **Cells and caption (<nowiki>| or ||, ! or !!, and |+</nowiki>) hold content. So separate any attributes from content with a single pipe (|). Cell content may follow on same line or on following lines. **Table and row marks (<nowiki>{| and |-</nowiki>) do not directly hold content. Do ''not'' add pipe (|) after their optional attributes. If you erroneously add a pipe after attributes for the table mark or row mark the parser will delete it ''and'' your final attribute if it was touching the erroneous pipe! *'''Content''' may (a) follow its cell mark on the same line after any optional XHTML attributes or (b) on lines below the cell mark. Content that uses wiki markup that itself needs to start on a new line, such as lists, headers, or nested tables, must of course be on its own new line. ==Simple table== ===Plain=== The following table lacks borders and good spacing but shows the simplest wiki markup table structure {| width="100%" |width="50%"| {| |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ===Alternative=== For more table-ish looking wiki markup cells can be listed on one line separated by ||. This does not scale well for longer cell content such as paragraphs. It works well for short bits of content however, such as our example table. Extra spaces within cells in the wiki markup can be added, as I have done in the wiki markup below, to make the wiki markup itself look better but they do not affect the actual table rendering. HTML attributes can be added to this table following the examples in other tables on this page but have been left out of the following example for simplicity. {| width="100%" |width="50%"| {| | Orange || Apple || more |- | Bread || Pie || more |- | Butter || Ice cream || and more |} |width="50%"| <pre> {| | Orange || Apple || more |- | Bread || Pie || more |- | Butter || Ice cream || and more |} </pre> |} ===With HTML attributes=== You can add HTML attributes to make your table look better ====border="1"==== {| width="100%" |width="50%"| {| border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ====align="center" border="1"==== {| width="100%" |width="50%"| {| align="center" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| align="center" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ====align="right" border="1"==== You can put attributes on individual '''cells.''' Numbers for example may look better aligned right {| width="100%" |width="50%"| {| border="1" |Orange |Apple |align="right"|12,333.00 |- |Bread |Pie |align="right"|500.00 |- |Butter |Ice cream |align="right"|1.00 |} |width="50%"| <pre> {| border="1" |Orange |Apple |align="right"|12,333.00 |- |Bread |Pie |align="right"|500.00 |- |Butter |Ice cream |align="right"|1.00 |} </pre> |} You can put attributes on individual '''rows,''' too. {| width="100%" |width="50%"| {| border="1" |Orange |Apple |align="right"|12,333.00 |- |Bread |Pie |align="right"|500.00 |- style="font-style:italic;color:green;" |Butter |Ice cream |align="right"|1.00 |} |width="50%"| <pre> {| border="1" |Orange |Apple |align="right"|12,333.00 |- |Bread |Pie |align="right"|500.00 |- style="font-style:italic;color:green;" |Butter |Ice cream |align="right"|1.00 |} </pre> |} ====cellspacing="0" border="1"==== {| width="100%" |width="50%"| {| cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ====cellpadding="20" cellspacing="0" border="1"==== {| width="100%" |width="50%"| {| cellpadding="20" cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| cellpadding="20" cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ===With HTML attributes and CSS styles=== CSS style attributes can be added with or without other HTML attributes ====style="color:green;background-color:#ffffcc;" cellpadding="20" cellspacing="0" border="1"==== {| width="100%" |width="50%"| {| style="color:green;background-color:#ffffcc;" cellpadding="20" cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| style="color:green;background-color:#ffffcc;" cellpadding="20" cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ==Table with TH headings== TH (HTML table headings) can be created by using ! instead of |. Headings usually show up bold and centered by default. ===Top headings=== ====Each column==== {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" !Yummy !Yummier |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" !Yummy !Yummier |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ====Colspan="2"==== {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" !colspan="2"|Yummies |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" ! colspan="2"|Yummies |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ===Side headings=== ====Default==== {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" !Fruit |Orange |Apple |- !Dish |Bread |Pie |- !Complement |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" !Fruit |Orange |Apple |- !Dish |Bread |Pie |- !Complement |Butter |Ice cream |} </pre> |} ====Right justify==== Right justified side headings can be done as follows {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" !align="right" |Fruit |Orange |Apple |- !align="right" |Dish |Bread |Pie |- !align="right" |Complement |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" !align="right" |Fruit |Orange |Apple |- !align="right" |Dish |Bread |Pie |- !align="right" |Complement |Butter |Ice cream |} </pre> |} ==Caption== A '''table caption''' can be added to the top of any table as follows {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" |+Food complements |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" |+Food complements |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} '''Attributes''' can be added to the caption as follows {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" |+align="bottom" style="color:#e76700;"|''Food complements'' |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" |+align="bottom" style="color:#e76700;"|''Food complements'' |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ==Table with H1, H2, H3 etc. headings== HTML H1, H2, H3, H4 etc. headings can be created the standard wiki markup way with ==equal== signs and '''must be on a line all by themselves''' to work. '''Preview the whole table.''' If you click on an edit tab for a heading ''within'' a table, edit, and preview, the parent table will display erroneously broken because part of it will be missing. Keep the heading hierarchy consistent with the rest of the page so that the table of contents at page top works correctly. {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" |colspan="2"| ===Yummiest=== |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" |colspan="2"| ===Yummiest=== |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ==Caveat== ===Negative numbers=== Negative value minus sign can break your table (it may display missing some values) if you start a cell on a new line with a negative number or a parameter that evaluates to a negative number (|-6) because that is the wiki markup for table row, not table cell. To avoid this, insert a space before the value (| -6) or use in-line cell markup (||-6). [[Category:Help|Tables]] [[Category:Tables]] 219e8f346e0cd7ca043e36b3bc5de04cefd5b9af 12 1421 1437 1513 2007-08-03T09:23:05Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} MediaWiki knows some defined variables which can be used in texts. They are rendered differently depending on the time, the site and the page. Their syntax is similar to templates. If you want a page to show always the current time, you use <code><nowiki>{{CURRENTTIME}}</nowiki></code>. To distinguish variables from templates, most variables are capitalized. <center> {| {{Prettytable}} |- !{{Hl3}}|'''Description''' !{{Hl3}}|'''You type''' !{{Hl3}}|'''You get''' |- ! {{Hl2}} colspan="3" | '''Date & Wiki variables''' |- | Current month |<nowiki>{{CURRENTMONTH}}</nowiki> |{{CURRENTMONTH}} |- | Current month |<nowiki>{{CURRENTMONTHNAME}}</nowiki> |{{CURRENTMONTHNAME}} |- | Genitive form<br>of current month name |<nowiki>{{CURRENTMONTHNAMEGEN}}</nowiki> |{{CURRENTMONTHNAMEGEN}} |- | Abbreviated month name |<nowiki>{{CURRENTMONTHABBREV}}</nowiki> |{{CURRENTMONTHABBREV}} |- |Day of the month |<nowiki>{{CURRENTDAY}}</nowiki> |{{CURRENTDAY}} |- |Day of the week |<nowiki>{{CURRENTDOW}}</nowiki> |{{CURRENTDOW}} |- |Day of the week |<nowiki>{{CURRENTDAYNAME}}</nowiki> |{{CURRENTDAYNAME}} |- | Week |<nowiki>{{CURRENTWEEK}}</nowiki> |{{CURRENTWEEK}} |- | Year |<nowiki>{{CURRENTYEAR}}</nowiki> |{{CURRENTYEAR}} |- | current time |<nowiki>{{CURRENTTIME}}</nowiki> |{{CURRENTTIME}} |- | number of pages in this Wiki (total) |<nowiki>{{NUMBEROFPAGES}}</nowiki> |{{NUMBEROFPAGES}} |- | numbers of articles in this Wiki |<nowiki>{{NUMBEROFARTICLES}}</nowiki> |{{NUMBEROFARTICLES}} |- | numbers of uploaded files |<nowiki>{{NUMBEROFFILES}}</nowiki> |{{NUMBEROFFILES}} |- ! {{Hl2}} colspan="3" | '''Namespaces''' |- | |<nowiki>{{ns:-2}} or {{ns:Media}}</nowiki> |{{ns:-2}} |- | |<nowiki>{{ns:-1}} or {{ns:Special}}</nowiki> |{{ns:-1}} |- | |<nowiki>{{ns:1}} or {{ns:Talk}}</nowiki> | {{ns:1}} |- | | <nowiki>{{ns:2}} or {{ns:User}}</nowiki> | {{ns:2}} |- | | <nowiki>{{ns:3}} or {{ns:User_talk}}</nowiki> | {{ns:3}} |- | | <nowiki>{{ns:4}} or {{ns:Project}}</nowiki> | {{ns:4}} |- | | <nowiki>{{ns:5}} or {{ns:Project_talk}}</nowiki> | {{ns:5}} |- | | <nowiki>{{ns:6}} or {{ns:Image}}</nowiki> | {{ns:6}} |- | | <nowiki>{{ns:7}} or {{ns:Image_talk}}</nowiki> | {{ns:7}} |- | | <nowiki>{{ns:8}} or {{ns:MediaWiki}}</nowiki> | {{ns:8}} |- | | <nowiki>{{ns:9}} or {{ns:MediaWiki_talk}}</nowiki> | {{ns:9}} |- | | <nowiki>{{ns:10}} or {{ns:Template}}</nowiki> | {{ns:10}} |- | | <nowiki>{{ns:11}} or {{ns:Template_talk}}</nowiki> | {{ns:11}} |- | | <nowiki>{{ns:12}} or {{ns:Help}}</nowiki> | {{ns:12}} |- | | <nowiki>{{ns:13}} or {{ns:Help_talk}}</nowiki> | {{ns:13}} |- | | <nowiki>{{ns:14}} or {{ns:Category}}</nowiki> | {{ns:14}} |- | | <nowiki>{{ns:15}} or {{ns:Category_talk}}</nowiki> | {{ns:15}} |- ! {{Hl2}} colspan="3" | '''Server specific''' |- | Sitename | <nowiki>{{SITENAME}}</nowiki> | {{SITENAME}} |- | Servername | <nowiki>{{SERVERNAME}}</nowiki> | {{SERVERNAME}} |- | Server | <nowiki>{{SERVER}}</nowiki> | {{SERVER}} |- |- | Script path | <nowiki>{{SCRIPTPATH}}</nowiki> | {{SCRIPTPATH}} |- | Local part of the URL<br> | <nowiki>{{localurl:pagename}}</nowiki> | {{localurl:pagename}} |- | | <nowiki>{{localurl:pagename|</nowiki>''query string''<nowiki>}}</nowiki> | {{localurl:pagename|''query string''}} |- ! {{Hl2}} colspan="3" | '''Page related''' |- | Pagename | <nowiki>{{PAGENAME}}</nowiki> | {{PAGENAME}} |- | Pagename<br>(URL encoded) | <nowiki>{{PAGENAMEE}}</nowiki> | {{PAGENAMEE}} |- | Namespace | <nowiki>{{NAMESPACE}}</nowiki> | {{NAMESPACE}} |- | Revision ID | <nowiki>{{REVISIONID}}</nowiki> | {{REVISIONID}} |} </center> ==External links== [http://meta.wikimedia.org/wiki/Help:Variable Help:Variable] [[Category:Help|Variables]] 6359ec43c64fd4526119002a92e3f325057850fa 12 1422 1439 1515 2007-08-03T09:23:05Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} ==Upload a file== # In the [[Help:Navigation#Sidebar|sidebar]], under '''toolbox''', click "Upload file" # Next to the '''Source filename''', click '''Browse''' to locate the file on your computer. # Change the '''Destination filename''' to something descriptive, if necessary. # Fill in the '''Summary''', if necessary. # Click the '''Upload file''' button. After you've uploaded some files, [[Help:Images|use them in your articles]]. [[Category:Help|Managing files]] 4c43300bcaf807c82bf3164f90ebbbb7da883fdc 12 1423 1441 1440 2007-08-03T09:23:05Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Clicking on the [[special:preferences|my preferences]] link in the upper right while logged in allows you to change your preferences. ==Date format== The following is normally rendered depending on preferences: <pre> [[2001-01-05]] (or [[2001]]-[[01-05]]) (with leading zeros) [[2001]] [[January 5]] ([[2001]] [[January 05]]) [[January 5]], [[2001]] ([[January 05]], [[2001]]) [[5 January]] [[2001]] ([[05 January]] [[2001]]) [[January 5]] ([[January 05]]) [[5 January]] ([[05 January]]) </pre> With your current preference setting on this project the seven are rendered as follows: #[[2001-01-05]] ([[2001]]-[[01-05]]) #[[2001]] [[January 5]] ([[2001]] [[January 05]]) #[[January 5]], [[2001]] ([[January 05]], [[2001]]) #[[5 January]] [[2001]] ([[05 January]] [[2001]]) #[[January 5]] ([[January 05]]) #[[5 January]] ([[05 January]]) The user-specified date format does not seem to work on the [[mw:]] wiki for links! It works in Recent Changes etc. == See also == * [[Help:Skins]] ==External Links== [http://meta.wikimedia.org/wiki/Help:Preferences Help:Preferences] [[Category:Help|Preferences]] 9990d65c68a40a80b4c83da7be9b495a70f31aab 12 1424 1443 1519 2007-08-03T09:23:06Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Clicking on the [[special:preferences|my preferences]] link in the upper right while logged in then click on the '''Skin''' button to change your skin. You can also preview the skin by clicking the '''(preview)''' links next to each skin. You can make changes to the current skin's stylesheet file (CSS) by creating a subpage of your userpage, "[[Special:Mypage/monobook.css|User:Yourname/monobook.css]]" for example. This requires your site admin to have enabled this feature -- if it is, you will see advice text at the top of your custom CSS page about clearing your browser's cache. == See also == * [[Help:Preferences]] == External Links == [http://meta.wikimedia.org/wiki/Skins Skins] [[Category:Help]] bdd8b61a54312b596a9613de1cc200211f8a599b 12 1425 1445 1521 2007-08-03T09:23:06Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Articles without a ':' in their name are in the main namespace. This namespace is for articles; documents that aren't directly related to this may go in other namespaces. When articles are in another namespace, they are specified as ''namespace'':''article''; i.e. this page is ''{{NAMESPACE}}'':''{{PAGENAME}}''. ;Help: Documentation about working with the wiki software. This could be mirrored from outside sites, or locally written. ;Image: For descriptions of uploaded files. You shouldn't create these directly; they are created when you click the [[special:upload|Upload file]] link in the toolbox. ;Media: Use this namespace to link to uploaded files directly, rather than through the description pages. ;MediaWiki: Use this namespace to change the default system messages, [http://meta.wikimedia.org/wiki/Help:System message See Help:System message on meta]. ;Project: Information about this wiki; i.e. policies that apply here. This namespace also has an alias, which is the name of the wiki installation. ;Talk: Each page has a corresponding ''discussion'' page. This can be used for feedback/comments about that page, or other local notes that another group may want to associate with the page, without modifying the document directly, or for any other additional information to associate with the document. ;Template: This is used for meta-information that is to be transcluded into multiple documents, such as tags to mark the status of a document. ;User: For personal notes. Each User has a corresponding user page for their own information. Users can also create subpages, by using a / after their name. ;User_talk: The discussion page on a user's page can be used for leaving messages. If this page is edited, the next time that user logs in they will see a box notifying them that they have new messages [[Category:Help|Namespaces]] ce056b03e0db36e8d30fd6d10d45a2fa01a85210 12 1426 1447 1523 2007-08-03T09:23:06Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} '''Special pages''' are pages that are created by the software on demand. They are located in their own [[Help:Namespaces|namespace]] ''Special:'' and are not editable directly as other pages. Some special pages depend on the [[Help:Preferences|preferences]] that have been set by a user, e.g. the number of titles which is displayed on a user's watchlist. ==List of special pages== Clicking the link ''[[Special:Specialpages]]'' will take you to a list of all special pages on a wiki. ==External links== [http://meta.wikimedia.org/wiki/Help:Special_page Help:Special page] [[Category:Help|Special pages]] [[Category:Special Pages| ]] 6d62eb7b20ac98315b2a1e21839b0d1c87ddabdb 12 1427 1449 1525 2007-08-03T09:23:07Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki '''Range blocks''' are technical restrictions applied through [[Special:Blockip]] to a group of [http://en.wikipedia.org/wiki/IP_addresses IP addresses] that prevent them from editing, creating new accounts, sending email through the wiki interface, et cetera. Registered accounts editing from these IP addresses will also be blocked, unless you check the box to only block anonymous editors. Range blocking is enabled on all [http://en.wikipedia.org/wiki/Wikimedia_Foundation Wikimedia] wikis; to enable it on other wikis, add "<code><nowiki>$wgSysopRangeBans = true;</nowiki></code>" in [http://www.mediawiki.org/wiki/LocalSettings.php LocalSettings.php]. ==How to block ranges== To block an IP range from [[Special:Blockip]], enter the first IP address in the range followed by a forward slash and a [http://en.wikipedia.org/wiki/Classless_inter-domain_routing Classless inter-domain routing] (CIDR) suffix. '''You should avoid performing range blocks unless you understand what you are doing, or potentially blocking tens of thousands of extra IP addresses is not a problem.''' ===Technical explanation=== CIDR notation is written as the IP address, a slash, and the CIDR affix (for example, the IPv4 "<code>10.2.3.41/24</code>" or IPv6 "<code>a3:b:c1:d:e:f:1:21/24</code>"). The CIDR affix is the number of starting digits every IP address in the range have in common when written in binary. For example: "<code>10.10.1.32</code>" is binary "<code>00001010.00001010.00000001.00100000</code>", so <code>10.10.1.32/27</code> will match the first 27 digits ("<code><u>00001010</u>.<u>00001010</u>.<u>00000001</u>.<u>001</u>00000</code>"). The IP addresses <code>10.10.1.32</code>–<code>10.10.1.63</code>, when converted to binary, all have the same 27 first digits and will be blocked if <code>10.10.1.32/27</code> is blocked. As the CIDR affix increases, the block affects less IP addresses (see [[#Table|table of example ranges]]). CIDR affixes are not the same for IPv4 addresses as they are for IPv6 addresses. ===Calculating the CIDR affix=== You can use the [[#Table|table of sample ranges]] below to guess the range, or manually calculate the range: # Convert the first and last IP addresses in the range to binary using [[#conversion_list|the list below]]. # Place both IP addresses one atop the other, and count how many starting digits are exactly alike. This is the CIDR affix. # Double-check! Being off by one digit could extend your block by thousands of addresses. The example below calculates the CIDR range between 69.208.0.0 and 69.208.0.255. Note that this is a simple example; some groups of IP addresses do not so neatly fit CIDR affixes, and need multiple different-sized blocks to block the exact range. IP addresses: 69.208.0.0 69.208.0.255 &nbsp; Convert to binary: 0100 0101.1101 0000.0000 0000.0000 0000 0100 0101.1101 0000.0000 0000.1111 1111 &nbsp; Count identical first numbers: '''0100 0101.1101 0000.0000 0000'''.0000 0000 '''0100 0101.1101 0000.0000 0000'''.1111 1111 |____________________________| 24 digits &nbsp; CIDR range: 69.208.0.0/24 </pre> ===<span id="conversion list">List of decimal->binary conversions</span>=== <div style="-moz-column-count:4; -moz-column-gap:2em; column-count:4; column-gap:2em; font-size:0.9em; font-family:monospace;"> Decimal -> Binary 0 -> 0000 0000 1 -> 0000 0001 2 -> 0000 0010 3 -> 0000 0011 4 -> 0000 0100 5 -> 0000 0101 6 -> 0000 0110 7 -> 0000 0111 8 -> 0000 1000 9 -> 0000 1001 10 -> 0000 1010 11 -> 0000 1011 12 -> 0000 1100 13 -> 0000 1101 14 -> 0000 1110 15 -> 0000 1111 16 -> 0001 0000 17 -> 0001 0001 18 -> 0001 0010 19 -> 0001 0011 20 -> 0001 0100 21 -> 0001 0101 22 -> 0001 0110 23 -> 0001 0111 24 -> 0001 1000 25 -> 0001 1001 26 -> 0001 1010 27 -> 0001 1011 28 -> 0001 1100 29 -> 0001 1101 30 -> 0001 1110 31 -> 0001 1111 32 -> 0010 0000 33 -> 0010 0001 34 -> 0010 0010 35 -> 0010 0011 36 -> 0010 0100 37 -> 0010 0101 38 -> 0010 0110 39 -> 0010 0111 40 -> 0010 1000 41 -> 0010 1001 42 -> 0010 1010 43 -> 0010 1011 44 -> 0010 1100 45 -> 0010 1101 46 -> 0010 1110 47 -> 0010 1111 48 -> 0011 0000 49 -> 0011 0001 50 -> 0011 0010 51 -> 0011 0011 52 -> 0011 0100 53 -> 0011 0101 54 -> 0011 0110 55 -> 0011 0111 56 -> 0011 1000 57 -> 0011 1001 58 -> 0011 1010 59 -> 0011 1011 60 -> 0011 1100 61 -> 0011 1101 62 -> 0011 1110 63 -> 0011 1111 64 -> 0100 0000 65 -> 0100 0001 66 -> 0100 0010 67 -> 0100 0011 68 -> 0100 0100 69 -> 0100 0101 70 -> 0100 0110 71 -> 0100 0111 72 -> 0100 1000 73 -> 0100 1001 74 -> 0100 1010 75 -> 0100 1011 76 -> 0100 1100 77 -> 0100 1101 78 -> 0100 1110 79 -> 0100 1111 80 -> 0101 0000 81 -> 0101 0001 82 -> 0101 0010 83 -> 0101 0011 84 -> 0101 0100 85 -> 0101 0101 86 -> 0101 0110 87 -> 0101 0111 88 -> 0101 1000 89 -> 0101 1001 90 -> 0101 1010 91 -> 0101 1011 92 -> 0101 1100 93 -> 0101 1101 94 -> 0101 1110 95 -> 0101 1111 96 -> 0110 0000 97 -> 0110 0001 98 -> 0110 0010 99 -> 0110 0011 100 -> 0110 0100 101 -> 0110 0101 102 -> 0110 0110 103 -> 0110 0111 104 -> 0110 1000 105 -> 0110 1001 106 -> 0110 1010 107 -> 0110 1011 108 -> 0110 1100 109 -> 0010 1101 110 -> 0110 1110 111 -> 0110 1111 112 -> 0111 0000 113 -> 0111 0001 114 -> 0111 0010 115 -> 0111 0011 116 -> 0111 0100 117 -> 0111 0101 118 -> 0111 0110 119 -> 0111 0111 120 -> 0111 1000 121 -> 0111 1001 122 -> 0111 1010 123 -> 0111 1011 124 -> 0111 1100 125 -> 0111 1101 126 -> 0111 1110 127 -> 0111 1111 128 -> 1000 0000 129 -> 1000 0001 130 -> 1000 0010 131 -> 1000 0011 132 -> 1000 0100 133 -> 1000 0101 134 -> 1000 0110 135 -> 1000 0111 136 -> 1000 1000 137 -> 1000 1001 138 -> 1000 1010 139 -> 1000 1011 140 -> 1000 1100 141 -> 1000 1101 142 -> 1000 1110 143 -> 1000 1111 144 -> 1001 0000 145 -> 1001 0001 146 -> 1001 0010 147 -> 1001 0011 148 -> 1001 0100 149 -> 1001 0101 150 -> 1001 0110 151 -> 1001 0111 152 -> 1001 1000 153 -> 1001 1001 154 -> 1001 1010 155 -> 1001 1011 156 -> 1001 1100 157 -> 1001 1101 158 -> 1001 1110 159 -> 1001 1111 160 -> 1010 0000 161 -> 1010 0001 162 -> 1010 0010 163 -> 1010 0011 164 -> 1010 0100 165 -> 1010 0101 166 -> 1010 0110 167 -> 1010 0111 168 -> 1010 1000 169 -> 1010 1001 170 -> 1010 1010 171 -> 1010 1011 172 -> 1010 1100 173 -> 1010 1101 174 -> 1010 1110 175 -> 1010 1111 176 -> 1011 0000 177 -> 1011 0001 178 -> 1011 0010 179 -> 1011 0011 180 -> 1011 0100 181 -> 1011 0101 182 -> 1011 0110 183 -> 1011 0111 184 -> 1011 1000 185 -> 1011 1001 186 -> 1011 1010 187 -> 1011 1011 188 -> 1011 1100 189 -> 1011 1101 190 -> 1011 1110 191 -> 1011 1111 192 -> 1100 0000 193 -> 1100 0001 194 -> 1100 0010 195 -> 1100 0011 196 -> 1100 0100 197 -> 1100 0101 198 -> 1100 0110 199 -> 1100 0111 200 -> 1100 1000 201 -> 1100 1001 202 -> 1100 1010 203 -> 1100 1011 204 -> 1100 1100 205 -> 1100 1101 206 -> 1100 1110 207 -> 1100 1111 208 -> 1101 0000 209 -> 1101 0001 210 -> 1101 0010 211 -> 1101 0011 212 -> 1101 0100 213 -> 1101 0101 214 -> 1101 0110 215 -> 1101 0111 216 -> 1101 1000 217 -> 1101 1001 218 -> 1101 1010 219 -> 1101 1011 220 -> 1101 1100 221 -> 1101 1101 222 -> 1101 1110 223 -> 1101 1111 224 -> 1110 0000 225 -> 1110 0001 226 -> 1110 0010 227 -> 1110 0011 228 -> 1110 0100 229 -> 1110 0101 230 -> 1110 0110 231 -> 1110 0111 232 -> 1110 1000 233 -> 1110 1001 234 -> 1110 1010 235 -> 1110 1011 236 -> 1110 1100 237 -> 1010 1101 238 -> 1110 1110 239 -> 1110 1111 240 -> 1111 0000 241 -> 1111 0001 242 -> 1111 0010 243 -> 1111 0011 244 -> 1111 0100 245 -> 1111 0101 246 -> 1111 0110 247 -> 1111 0111 248 -> 1111 1000 249 -> 1111 1001 250 -> 1111 1010 251 -> 1111 1011 252 -> 1111 1100 253 -> 1111 1101 254 -> 1111 1110 255 -> 1111 1111 </div> ===<span id="Table">Table of sample ranges</span>=== The table below shows the IP blocks each CIDR suffix affects. Note that MediaWiki only supports blocking CIDR suffixes 16&ndash;32. {| class="prettytable" ! CIDR ! Start Range ! End Range ! Total addresses ! Bits selected in IP address |- style="color:gray;" | 69.208.0.0'''/0''' | 0.0.0.0 | 255.255.255.255 | 4,294,967,296 | ********.********.********.******** |- style="color:gray;" | 69.208.0.0'''/1''' | 0.0.0.0 | 127.255.255.255 | 2,147,483,648 | 0*******.********.********.******** |- style="color:gray;" | 69.208.0.0'''/4''' | 65.0.0.0 | 79.255.255.255 | 268,435,456 | 0100****.********.********.******** |- style="color:gray;" | 69.208.0.0'''/8''' | 69.0.0.0 | 69.255.255.255 | 67,108,864 | 01000101.********.********.******** |- style="color:gray;" | 69.208.0.0'''/11''' | 69.208.0.0 | 69.238.255.255 | 2,197,152 | 01000101.110*****.********.******** |- style="color:gray;" | 69.208.0.0'''/12''' | 69.208.0.0 | 69.223.255.255 | 1,048,576 | 01000101.1101****.********.******** |- style="color:gray;" | 69.208.0.0'''/13''' | 69.208.0.0 | 69.215.255.255 | 524,288 | 01000101.11010***.********.******** |- style="color:gray;" | 69.208.0.0'''/14''' | 69.208.0.0 | 69.211.255.255 | 262,144 | 01000101.110100**.********.******** |- style="color:gray;" | 69.208.0.0'''/15''' | 69.208.0.0 | 69.209.255.255 | 131,072 | 01000101.1101000*.********.******** |- | 69.208.0.0'''/16''' | 69.208.0.0 | 69.208.255.255 | 65,536 | 01000101.11010000.********.******** |- | 69.208.0.0'''/17''' | 69.208.0.0 | 69.208.127.255 | 32,768 | 01000101.11010000.0*******.******** |- | 69.208.0.0'''/18''' | 69.208.0.0 | 69.208.63.255 | 16,384 | 01000101.11010000.00******.******** |- | 69.208.0.0'''/19''' | 69.208.0.0 | 69.208.31.255 | 8,192 | 01000101.11010000.000*****.******** |- | 69.208.0.0'''/20''' | 69.208.0.0 | 69.208.15.255 | 4,096 | 01000101.11010000.0000****.******** |- | 69.208.0.0'''/21''' | 69.208.0.0 | 69.208.7.255 | 2,048 | 01000101.11010000.00000***.******** |- | 69.208.0.0'''/22''' | 69.208.0.0 | 69.208.3.255 | 1,024 | 01000101.11010000.000000**.******** |- | 69.208.0.0'''/23''' | 69.208.0.0 | 69.208.1.255 | 512 | 01000101.11010000.0000000*.******** |- | 69.208.0.0'''/24''' | 69.208.0.0 | 69.208.0.255 | 256 | 01000101.11010000.00000000.******** |- | 69.208.0.0'''/25''' | 69.208.0.0 | 69.208.0.127 | 128 | 01000101.11010000.00000000.0******* |- | 69.208.0.0'''/26''' | 69.208.0.0 | 69.208.0.63 | 64 | 01000101.11010000.00000000.00****** |- | 69.208.0.0'''/27''' | 69.208.0.0 | 69.208.0.31 | 32 | 01000101.11010000.00000000.000***** |- | 69.208.0.0'''/28''' | 69.208.0.0 | 69.208.0.15 | 16 | 01000101.11010000.00000000.0000**** |- | 69.208.0.0'''/29''' | 69.208.0.0 | 69.208.0.7 | 8 | 01000101.11010000.00000000.00000*** |- | 69.208.0.0'''/30''' | 69.208.0.0 | 69.208.0.3 | 4 | 01000101.11010000.00000000.000000** |- | 69.208.0.0'''/31''' | 69.208.0.0 | 69.208.0.1 | 2 | 01000101.11010000.00000000.0000000* |- | 69.208.0.0'''/32''' | 69.208.0.0 | 69.208.0.0 | 1 | 01000101.11010000.00000000.00000000 |} ==References== * [http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing Classless Inter-Domain Routing] * [http://www.ccci.com/tools/subcalc/binary.html Converting IP addresses to binary] ==External links== * [http://apps.csc.fi/laskin2.html Netmask calculator] which helps in making the correct decision for range blocks. {{PD Help Page}} [[Category:Help|Range blocks]] dd441b70a5ca16be4ed43f1c05674eb02e0b3143 12 1428 1451 1527 2007-08-03T09:23:07Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} [http://meta.wikimedia.org/wiki/Help:User_levels Please see this page on meta.] 65998a13c8fd409fb9002b6d933bdd39c6ec47a8 12 1429 1453 1529 2007-08-03T09:23:07Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} You can obtain a copy of the help system in this wiki for local use in your own wiki following these steps: * Step one: In a separate browser window, navigate to [[Special:Export]]. * Step two: You will see an open textbox asking you for a list of pages to export. * Step three: Copy the list below and paste it into that textbox. This is a generally useful set, making the resulting help structure in your wiki look almost identical to the one here. [[Help:Contents]] [[Help:Navigation]] [[Help:Searching]] [[Help:Tracking_changes]] [[Help:Editing_pages]] [[Help:Editing]] [[Help:Starting_a_new_page]] [[Help:Formatting]] [[Help:Links]] [[Help:Categories]] [[Help:Images]] [[Help:Templates]] [[Help:Tables]] [[Help:Variables]] [[Help:Managing_files]] [[Help:Preferences]] [[Help:Skins]] [[Help:Namespaces]] [[Help:Special pages]] [[Help:Range_blocks]] [[Help:Managing_user_rights]] [[Help:Copying]] [[Category:Help]] [[Category:Category]] [[Template:PD_Help_Page]] [[Template:Admin_tip]] [[Template:Prettytable]] [[Template:Hl2]] [[Template:Hl3]] [[Template:Thankyou]] [[:Image:Example.jpg]] [[:Image:Tools.png]] [[:Image:M-en-sidebar.png]] [[:Image:M-en-pagetabs.png]] [[:Image:M-en-userlinks.png]] [[:Image:M-en-recentchanges.png]] Sysops only: [[Template:Meta]] [[Template:Click]] * Step four: Make sure the box "Include only the current revision, not the full history" is CHECKED. * Step five: Click the 'Export' button. * Step six: Save the file to your desktop or other convenient location you'll remember. * Step seven: Navigate to Special:Import on YOUR wiki. (You must be logged in as Admin/Sysop level to do this.) * Step eight: Browse for the file you saved, and click 'Upload'. * Step nine: Download each of the Image files mentioned above (the export/import only copies the image descriptions). * Step ten: Upload each of the Images to the new wiki. * Step eleven: If they haven't come across, manually edit <nowiki>[[Category:Help]]</nowiki> and <nowiki>[[Category:Category]]</nowiki>. Although not complete, at least this gets you started on a decent help system. 3e6e01455883843be45830cbc6365b00b17b8b1d 10 1430 1455 1531 2007-08-03T09:23:08Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki <div style="color:#000000; border:solid 1px #A8A8A8; padding:0.5em 1em 0.5em 0.7em; margin:0.5em 0.5em 0.5em 0.5em; background-color:#FFFFFF;font-size:95%; vertical-align:middle;"> {| | These pages are derived from the [http://www.mediawiki.org/wiki/Help:Contents public domain help pages] at [http://www.mediawiki.org/wiki/MediaWiki MediaWiki]. See [[Help:Contents]] for an overview of all Help pages. |} 01fceb84165358212c4e090fed0ad9f20db091d6 10 1431 1457 1533 2007-08-03T09:23:08Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki <onlyinclude><div style="font-color:#535068; border:solid 1px #A8A8A8; padding:0.5em 1em 0.5em 0.7em; margin:0.5em 0em; background-color:#FFFFFF;font-size:85%;"> {| |[[Image:Tools.png|20px]] |'''Tip for wiki admins:''' {{{tip}}} |} </div></onlyinclude> [[Category:Info templates|{{PAGENAME}}]] 0221001a20efe5e58a5aaf59da0c13a5b59992ff 10 1432 1459 1535 2007-08-03T09:23:08Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki border="2" cellspacing="4" cellpadding="3" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse;empty-cells:show;{{{style|}}}" <noinclude>[[Category:Utility templates|{{PAGENAME}}]]</noinclude> 66042a0bfed4d0e2d45f4bb132266235062a9aa7 10 1433 1461 1537 2007-08-03T09:23:08Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki bgcolor="#A7C1F2" <noinclude>[[Category:Utility templates|{{PAGENAME}}]]</noinclude> 3afc2b03bfd3f4c68c0fbb45b7ef52d642b0f8df 6 1436 1467 1543 2007-08-03T09:23:09Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki ---- Note: This image is used as the example image when you use the edit toolbar (English default installation of MediaWiki). ---- * '''Description:''' Sunflowers in Fargo, North Dakota, U.S. * '''Source:''' [http://www.ars.usda.gov/is/graphics/photos/k5751-1.htm www.ars.usda.gov] (cropped and reworked a bit) * '''Author:''' Bruce Fritz * '''Licensing:''' ''This image is in the '''public domain''' because it contains materials that originally came from the Agricultural Research Service, the research agency of the United States Department of Agriculture.'' 3bb0a024af1a31b13b67e93e685dabf8801ac14e 6 1437 1469 1468 2007-08-03T09:23:09Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki da39a3ee5e6b4b0d3255bfef95601890afd80709 1482 1469 2007-08-03T09:27:02Z NJAEWikiSysop 1 wikitext text/x-wiki Derived from http://www.mediawiki.org/wiki/Image:Tools.svg 11a8b6c88f2197f90475aee334470507537f5c39 6 1438 1471 1547 2007-08-03T09:23:09Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki From http://www.mediawiki.org/wiki/Image:M-en-sidebar.png . Released under version 2 of the [http://www.gnu.org/copyleft/gpl.html GNU General Public License]. f833d106ffae6bc69bdafb18fc38df5b710fe289 6 1439 1473 1549 2007-08-03T09:23:09Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki From http://www.mediawiki.org/wiki/Image:M-en-pagetabs.png . Released under version 2 of the [http://www.gnu.org/copyleft/gpl.html GNU General Public License]. cce112f6ad6d1e761078b8be7decf646ab411fd2 6 1440 1475 1551 2007-08-03T09:23:09Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki From http://www.mediawiki.org/wiki/Image:M-en-userlinks.png bb7614d78c623b8b671c6f3ed4c33eda2847683e 6 1441 1477 1553 2007-08-03T09:23:10Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki From http://www.mediawiki.org/wiki/Image:M-en-recentchanges.png 062b0298ecff4e69cf8c61471fd2abfc76d251b2 10 1443 1481 1480 2007-08-03T09:23:10Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki <includeonly><div style="position: relative; width: {{{width}}}; height: {{{height}}}; overflow: hidden;"><div style="position: absolute; top: 0px; left: 0px; font-size: 100px; overflow: hidden; line-height: 100px; z-index: 3;">[[{{{link}}}|&nbsp;&nbsp;&nbsp;]]</div> <div style="position: absolute; top: 0px; left: 0px; z-index: 2;">[[Image:{{{image}}}|{{{width}}}|{{{link}}}]]</div> </div></includeonly><noinclude> '''Usage:''' <nowiki>{{Click | image=myimage.png | link=PageName | width=10px | height=10px}}</nowiki> This template attempts to superimpose an invisible link on an image, such that clicking on the image does ''not'' take you to the image description page (normal behaviour). It's useful for small icons/logos which are representative of an article, and which a user might click on expecting to be taken to that article. Here we are using it on the [[Main Page]] for example. It's use should minimised because of the following problems. ==== Problems ==== It doesn't work in text-only browsers, and in screen readers for the disabled, and possibly other situations. The technique of using [[w:CSS|CSS]] to change page content also completely breaks an article's [[w:web accessibility|web accessibility]] by contravening a [[w:WAI|WAI]] priority-one checkpoint.[http://www.w3.org/TR/WAI-WEBCONTENT/#tech-order-style-sheets] [[Extension:ImageMap]] can achieve the same effect and much more. ==== See also ==== * [http://en.wikipedia.org/wiki/Template:Click Template:Click on wikipedia] where we got this from. * [[Mediazilla:539|<span title="ASSIGNED: Allow images that link somewhere other than the image page">Bug 539</span>]] - Feature request to allow linked images without nasty CSS. </noinclude> b9abc4147efc676be55fc03191b09e7f744ff313 14 1444 1483 2007-08-03T09:29:59Z NJAEWikiSysop 1 wikitext text/x-wiki '''Category:Help''' is for generic site user help. Only pages in namespace "Help:" should be in this category. ff9ce61ad5023a4a22cdfcd5b0b6764f2e48a652 14 1445 1484 2007-08-03T09:30:07Z NJAEWikiSysop 1 wikitext text/x-wiki [[Category:Category]] 6c67f71958e3af74b04ec83717abe026bc25faef 12 1409 1488 1411 2007-08-24T08:40:49Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} '''Reading''' : [[Help:Navigation|Navigation]] : [[Help:Searching|Searching]] : [[Help:Tracking changes|Tracking changes]] '''Editing''' : [[Help:Editing pages|Editing pages]] : [[Help:Starting a new page|Starting a new page]] : [[Help:Formatting|Formatting]] : [[Help:Links|Links]] : [[Help:Categories|Categories]] : Advanced editing: :: [[Help:Images|Images]] :: [[Help:Tables|Tables]] :: [[Help:Templates|Templates]] :: [[Help:Variables|Variables]] :: [[Help:Managing files|Managing files]] : Personal customizing: :: [[Help:Preferences|Preferences]] :: [[Help:Skins|Skins]] : Wiki administrator tools: :: [[Help:Range blocks|Range IP blocks]] '''Server administration''' : [[Help:Managing user rights|Managing user rights]] : [[Help:Namespaces|Namespaces]] : [[Help:Copying|Copying 'Help']] ==See also== [http://meta.wikimedia.org/wiki/Help:Displaying_a_formula Mathematical formula] [[Category:Help| ]] 764bb7e4a47850777b11ff22c0cecdf1f26457d3 12 1410 1490 1413 2007-08-24T08:40:49Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} If you look at a page in MediaWiki, you'll find three main navigation elements: The sidebar on the left gives you access to important pages in the Wiki like recent changes or file upload. At the top of the page are the links which belong to the page currently displayed: its associated discussion page, the version history, and - most notably - the edit link. In the top right corner you'll find, as an anonymous user, the link to create an account and login (they are on the same page). As a logged-in user you have a collection of personal links, like the one to your user page or your preferences. == Sidebar == [[Image:M-en-sidebar.png|framed|right|Example sidebar, shown on the left of the page]] The sidebar is displayed on the left edge of the page below the site logo (if using the default MonoBook skin). This sidebar gives you access to important pages in the Wiki like recent changes or file upload. === Navigation === A click on the logo brings you back to the start page of the wiki. The links take you to important pages in the wiki, they can be configured by site administrators of the individual wikis {{Admin tip|tip=You can customize the links in the navigation section via [[Special:Allmessages]]. For example edit [[MediaWiki:Sitesupport]] (displayed name) and [[MediaWiki:Sitesupport-url]] (target page). Or create a menu on your own in [[MediaWiki:Sidebar]].}} === Toolbox === The toolbox contains a selection of links which change depending on what type of page you are viewing. On all pages (except special pages): * ''What links here'' takes you to a special page that lists the pages on this wiki which contain a link to the current page. This is helpful when you are looking for pages of related information. The ''What links here'' information can also be useful when you are refactoring wiki pages and need to check whether links to this page are still relevant after changes in the current page. * The ''Related changes'' tool lists all recent changes in the pages linked to from the current page. Recent changes to all relevant template pages are included in the resulting page list. The "Hide minor edits" option that can be set in the user [[Help:Preferences|preferences]] applies, among other things, to ''Related Changes''. On all pages (including special pages): * ''Upload file'' displays a special page that allows logged-in users to upload images and other files to the wiki. Uploaded files can be linked-from or embedded-in wiki pages. Uploading files, viewing files on the server, including them in wiki pages and managing the uploaded files is discussed in the [[Help:Managing files|managing files]] section of this manual. This is not displayed if file uploading has been disabled or not enabled in the first place. {{Admin tip|tip=To enable file uploading someone with access to the MediaWiki installation files needs to edit the <code>LocalSettings.php</code> file and uncomment or add the option <code>$wgEnableUploads = true;</code>. Uploaded files will be stored in the images folder specified by the <code>$wgUploadPath</code> variable in the <code>LocalSettings.php</code>. This directory must be writable if file uploads is enabled. The [http://www.mediawiki.org/wiki/Manual:LocalSettings.php#Upload_location upload location] and [http://www.mediawiki.org/wiki/Manual:LocalSettings.php#Image_uploads image uploads]] settings are described in more detail on the MetaWiki [http://www.mediawiki.org/wiki/Manual:LocalSettings.php LocalSettings.php] page.}} * The ''Special pages'' tool lists the MediaWiki special pages. In MediaWiki terminology, a special page is one that presents information about the Wiki and/or allows access to administration activities for the wiki. For example, a list of users registered with the wiki, statistics about the wiki such as the number of pages and number of page edits, system logs, a list of orphaned pages, and so on. These special pages are commonly generated when the special page is loaded rather than being stored in the wiki database. :''The function and use of the default special pages can be found in the [[Help:Special pages|special pages]] section of this manual.'' == Page Tabs == [[Image:M-en-pagetabs.png|framed|right|Default page tabs at the top of the page]] The page tabs are displayed at the top of the article to the right of the site logo (if using the default MonoBook skin). These tabs allow you to perform actions or view pages that are related to the current article. The available default actions include: viewing, editing, and discussing the current article. The specific tabs displayed on your pages depend on whether or not you are logged into the wiki and whether you have sysop (administrator) privileges on the wiki. On special pages only the namespace tab is displayed. ;Default for all users : ''[[Help:Namespaces|namespace]]'' (article, help, special page, template, user page etc.) : ''discussion'' : ''edit'' (may be view source if anonymous editing is enabled, the page is in the MediaWiki namespace or the page is protected) : ''history'' ;Extra tabs for logged in users: : ''move'' : ''watch'' ;Extra tabs for sysops: : ''protect'' : ''delete'' Extra tabs may be added, or some taken away, by wiki administrators using JavaScript or adding extensions so these may be different depending on which wiki you are using. == User Links == [[Image:M-en-userlinks.png|framed|right|Default user links at the top right of the page]] The user links are displayed at the top far right of the article (if using the default MonoBook skin). These tabs allow the logged-in user to view and edit their user page and wiki preferences. Additionally, the user links allow the user to quickly access their contributions to the wiki and logout. For anonymous users the user links is replaced by a link to the wiki login page or, if enabled, a link to your ip address and your ip address's talk page. * ''<username>'' *: This links to your user page which is where you can put information about yourself, store bits of information you want to remember or whatever else you fancy. * ''my talk'' *: This links to your discussion page, where people can leave messages for you. * ''preferences'' *: Allows you to change your personal site preferences. * ''my watchlist'' *: A list of all pages that you are watching. Pages can be added to this list by clicking 'watch' at the top of the page. * ''my contributions'' *: A list of all contributions you have made to the wiki. * ''log out'' *: Click this link to log out of the wiki. [[Category:Help|Navigation]] dac0d834e7134c30b4774792355b47a28eaaacfa 12 1411 1492 1415 2007-08-24T08:40:49Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} The quickest way to find information in MediaWiki is to look it up directly. On the left-hand side of your screen there is a '''Search''' box with two buttons under it labeled "Go" and "Search". <div style="border:1px solid #ababff; background: #fdfdff; padding: 1em; margin: 1.1em 1em 1em 0em; font-size:105%"> Put your keyword in the searchbox. * <font face="MS Sans Serif,Arial">'''Go'''</font> - (or ''Enter'' on keyboard) will take you automatically to the article, if it exists. * <font face="MS Sans Serif,Arial">'''Search'''</font> - will search the text of all pages on the wiki (with some restrictions, see below). </div> If you clicked 'search', or you clicked 'go' and the page you were looking for did not exist, you will be presented with a list of articles that matched your search criteria (or a message that no matches were found). == How it works == Here's how the search works: * Only the article content is searched - the page title is ignored. * The article content is searched in its raw (wikitext) form - i.e. it searches the text that appears in the edit box when you click 'edit', not the rendered page. This means that content coming from an included template will not be picked up, but the target of piped links will be. * Even if you enclose a phrase in quotes, the search looks for each word individually. e.g. if you enter "world war 2" it will return pages that contain "world" and "war" and "2". * The search is not case-sensitive, so "MediaWiki", "mediawiki" and "MEDIAWIKI" all give the same result. == Restricting the search == If you click the 'Search' button without filling in anything, you will be taken to 'Special:Search' which gives you extra searching options (also available from any search results list) You may find it useful to restrict a search to pages within a particular [[Help:Namespaces|namespace]] e.g. only search within the 'Talk' pages. Tick the namespaces you require for this search. By default only the [[Help:Namespaces|namespaces]] specified in your [[Help:Preferences|preferences]] will be searched. Logged-in users can change their preferences to specify the namespaces they want to search by default. __NOTOC__ [[Category:Help|Searching]] ac95b57af14452387e5cce87b4e1d3331328d642 12 1412 1494 1417 2007-08-24T08:40:49Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} [[Image:M-en-recentchanges.png|thumb|370px|'''Recent changes''' shows the latest changes to articles, file uploads, deletions and page moves]] MediaWiki offers a collection of special pages and tools to keep track of what is going on in the wiki. So you can watch for example: * recent changes to all pages * newly created pages * the contributions of one specific user * the revision history: all changes made to one page The most interesting special page is '''[[Special:Recentchanges]]'''. It displays all edits, file uploads, page moves, deletions and other actions done in the wiki. In the menu on top it offers a collection of links to customize your display: limit the number of changes shown, the number of days or restrict it to edits to a certain namespace. You can also hide edits marked as minor (don't forget that major changes can be flagged by a user as minor anyway). One line in recentchanges consists of several links: * ''diff'' displays the difference to the previous revision of the page, * ''hist'' links to the revision history of the page, * the link with the full title of the page brings you to the current version. If the title is in bold, it indicates that it is on your watchlist. Next is a flag describing the article modification type: *''N'' signalizes a new page, *''m'' a minor edit, *''b'' an edit made by a bot. Following the timestamp the user is mentioned with a link to his user and talk page and to his contributions. In italic follows a comment if the user has submitted one. === See also === * [[Special:Newpages]] * Related changes: [[Special:Recentchangeslinked/Pagename]] * User contributions: [[Special:Contributions/User]] * History * [[Special:Watchlist]] {{Admin tip|tip=You may want to add important pages in your wiki to [[MediaWiki:Recentchangestext]], the intro shown at [[Special:Recentchanges]]}} [[Category:Help|Tracking changes]] 2ea46d15eb1f30c7150a890d2edb9ba3a8dbe041 12 1413 1496 1419 2007-08-24T08:40:49Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Easy editing of pages is what wikis are made for. It only takes a few clicks. == Editing new pages == If the page you wish to edit does not exist yet, see [[Help:Starting a new page]] == Editing existing pages == * Click the '''edit''' [[Help:Navigation#Page Tabs|page tab]] * Make changes to the text in the edit box. Make sure you write clearly and concisely, and make sure your edit does something to improve the page. If you're making normal changes to the text like fixing spelling mistakes or grammar, inserting new sentences, etc, then you don't have to worry too much about formatting. When you do need to use some type of formatting, you do it using wiki syntax, see [[Help:Formatting]] for some of the common types of formatting used. * Optionally, enter a short note in the '''Summary''' box describing your changes. * Optionally preview your changes with the '''Preview''' button. * Click the '''Save page''' button. * One can rename pages as well See [http://meta.wikimedia.org/wiki/Help:Moving_a_page Help:Moving_a_page] === Discussion === Every article has its own discussion page where you can ask questions, make suggestions, or discuss corrections. Click the '''discussion''' [[Help:Navigation#Page Tabs|page tab]] to reach it. You can sign your message by writing 4 tildes (<nowiki>~~~~</nowiki>). The wiki software turns them into the current time and your username or your current IP address if you are not logged in. == External Links == [http://meta.wikimedia.org/wiki/Help:Editing Help:Editing] [[Category:Help|Editing pages]] 80611681e1843d02ef23a0e6a8b88f77f9f09660 12 1405 1498 1421 2007-08-24T08:40:49Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT[[Help:Editing pages]] 8640847489ad8b1546c4043eaae6daf05ecedd3c 12 1414 1500 1499 2007-08-24T08:40:50Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} There are several ways to start a new page. == Using Wikilinks == MediaWiki makes it very easy to link wiki pages using a standard syntax (see [[Help:Links|Links]]). If you (or anyone else) creates a link to an article that doesn't exist yet, the link will be coloured red, <span style="color: #ba0000">like this</span>. Clicking a red link, will take you to the edit page for the new article. Simply type your text, click save and the new page will be created. Once the page has been created, the link will change from <span style="color: #ba0000">red</span> to <span style="color: #002bb8;">blue</span> (<span style="color: #5a3696;">purple</span> for pages you've visited) indicating that the article now exists. == Using the URL == You can use the wiki's URL for creating a new page. The URL to an article of the wiki is usually something like this: *<code><nowiki>http://www.example.net/index.php/</nowiki>'''ARTICLE'''</code> &nbsp;&nbsp; or *<code><nowiki>http://www.example.net/wiki/</nowiki>'''ARTICLE'''</code> If you replace <code>'''ARTICLE'''</code> with the name of the page you wish to create, you will be taken to a blank page which indicates that no article of that name exists yet. Clicking the "''edit''" [[Help:Navigation#Page Tabs|page tab]] at the top of the page will take you to the edit page for that article, where you can create the new page by typing your text, and clicking submit. == From the search page == If you search for a page that doesn't exist (using the search box and 'go' button on the left of the page) then you will be provided with a link to create the new page. (Note that this technique doesn't work if you use the 'search' button). == Using the Inputbox extension == {{Admin tip|tip=You need to download and install the "inputbox extension" for this method to work.}} If the Inputbox extension is available on the wiki you can start a new page by typing in your article's name and clicking the "create article" button, as in the example below. {| align="center" style="border:1px solid black" width="60%" | <inputbox> type=create width=45 </inputbox> |} == Create redirects to your new page == Don't forget to setup redirects when you create a page. If you think another person may search for the page you've created by using a different name or spelling, please create the proper redirect(s). To create a redirect, the '''first line''' of the redirect page should read <code><nowiki>#REDIRECT [[Destination]]</nowiki></code>, where <code>Destination</code> is the page to which people should be redirected. This must appear as the very first line of the page. {{Admin tip|tip= If desired, once you create your new page, you should protect the page and allow only the type of users you would like to edit the page. }} [[Category:Help|Starting a new page]] 0c51b3590ad784835a3bb7f56042d300b9f998a8 12 1415 1502 1425 2007-08-24T08:40:50Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} You can format your text using wiki markup. This consists of normal characters like asterisks, single quotes or equation marks which have a special function in the wiki, sometimes depending on their position. For example, to format a word in ''italic'', you include it in two single quotes like <tt><nowiki>''this''</nowiki></tt> == Text formatting markup == {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Description''' |{{Hl2}} |'''You type''' |{{Hl2}} |'''You get''' |- | colspan="3" align="center" | ''applies anywhere'' |- |Italic text |<tt><nowiki>''italic''</nowiki></tt> |''italic'' |- | Bold text |<tt><nowiki>'''bold'''</nowiki></tt> |'''bold''' |- | Bold and italic |<tt><nowiki>'''''bold & italic'''''</nowiki></tt> |'''''bold & italic''''' |- |Escape wiki markup |<tt><nowiki><nowiki>no ''markup''</nowiki></nowiki></tt> |<nowiki>no ''markup''</nowiki> |- | colspan="3" align="center" | ''only at the beginning of the line'' |- |Headings of<br /> different sizes | <tt><nowiki>==level 1==</nowiki></tt><br /> <tt><nowiki>===level 2===</nowiki></tt><br /> <tt><nowiki>====level 3====</nowiki></tt><br /> <tt><nowiki>=====level 4=====</nowiki></tt> | ==Level 1== ===Level 2=== ====Level 3==== =====Level 4===== |- |Horizontal rule | <tt><nowiki>----</nowiki></tt> | ---- |- |Bullet list | &#42; one<br> &#42; two<br> &#42; three<br> &#42;&#42; three and one-third<br> &#42;&#42; three and two-thirds<br> | * one * two * three ** three and one-third ** three and two-thirds |- |Numbered list | &#35; one<br /> &#35; two<nowiki><br></nowiki>spanning several lines<nowiki><br></nowiki>without breaking the numbering<br /> &#35; three<br /> &#35;&#35; three point one<br /> &#35;&#35; three point two<br /> | # one # two<br>spanning several lines<br>without breaking the numbering # three ## three point one ## three point two |- | Mixture of bulleted<br /> and numbered lists | &#35; one<br /> &#35; two<br /> &#35;&#42; two point one<br /> &#35;&#42; two point two<br /> | # one # two #* two point one #* two point two |- |Definition list | &#59;Definition<br /> &#58;item 1<br /> &#58;item 2<br /> | ;Definition :item 1 :item 2 |- |Preformatted text | &nbsp;&nbsp;preformatted text is done with<br /> &nbsp;&nbsp;a space at the<br /> &nbsp;&nbsp;beginning of the line<br /> | preformatted text is done with a space at the beginning of the line |} == Paragraphs == MediaWiki ignores normal line breaks. To start a new paragraph, leave an empty line. You can also start a new line with the HTML tag <tt><nowiki><br&nbsp;/></nowiki></tt>. == HTML == Some HTML-Tags are allowed in MediaWiki, for example <tt><nowiki><code></nowiki></tt>, <tt><nowiki><div></nowiki></tt>, <tt><nowiki><span></nowiki></tt> and <tt><nowiki><font></nowiki></tt>. {{Admin tip|tip=If you trust your users you can allow full HTML by setting <code>$wgRawHtml = true;</code> in LocalSettings.php.}} == Other formatting == Beyond the text formatting markup shown above, here are some other formatting references: *[[Help:Links|Links]] *[[Help:Images|Images]] *[[Help:Tables|Tables]] [http://en.wikipedia.org/wiki/Wikipedia:WikiProject_User_Page_Help/Do-It-Yourself/Formatting_Text#Underlining More Formatting from Wikipedia] == External Links == [http://meta.wikimedia.org/wiki/Help:Editing Help:Editing] __NOTOC__ __NOEDITSECTION__ [[Category:Help|Formatting]] ed8bc65ec07575e2da066adadb871a7e0b367dec 12 1416 1504 1503 2007-08-24T08:40:51Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} There are four sorts of links in MediaWiki: # internal links to other pages in the wiki # external links to websites # inter-wiki links (links to other wikis) # inter-language links to other language versions of the same wiki To add an internal link, enclose the name of the page you want to link to in double square brackets. When you save the page, you'll see the new link pointing to your page. If the page exists already, it is displayed in blue, empty pages are displayed in red. Selflinks to the current page are not transformed in URLs but displayed in bold. The first letter of the target page is automatically capitalized and spaces are represented as underscores (typing an underscore in the link will have a similar effect as typing a space, but is not recommended, since the underscore will also be shown in the text). == How to link == {| {{Prettytable}} |- |{{Hl3}}|'''Description''' |{{Hl3}}|'''You type''' |{{Hl3}}|'''You get''' |- |Internal link |<tt><nowiki>[[Main Page]]</nowiki></tt> |[[Main Page]] |- |Category link |<tt><nowiki>[[:Category:Help]]</nowiki></tt> |[[:Category:Help]] |- |Piped link |<tt><nowiki>[[Main Page|different text]]</nowiki></tt> |[[Main Page|different text]] |- |Anchor link |<tt><nowiki>[[#External links|Anchor link]]</nowiki></tt> |[[#External links|Anchor link]] |- |External link |<tt><nowiki>http://mediawiki.org</nowiki></tt> |http://mediawiki.org |- |External link from internal image |<tt><nowiki>[http://targetURL {{Server}}/wiki/images/imagelocation</nowiki></tt> |Internal image |- |External link,<br /> different title |<tt><nowiki>[http://mediawiki.org MediaWiki]</nowiki></tt> |[http://mediawiki.org MediaWiki] |- |External link,<br /> unnamed |<tt><nowiki>[http://mediawiki.org]</nowiki></tt> |[http://mediawiki.org] |- |External link,<br /> same host unnamed |<tt><nowiki>[http://{{SERVERNAME}}/pagename]</nowiki></tt> |[http://{{SERVERNAME}}/pagename] |- |Interwiki link |<tt><nowiki>[[Wikipedia:MediaWiki]]</nowiki></tt> |Wikipedia:MediaWiki (note: this doesn't work on this wiki) |- |mailto |<tt><nowiki>mailto:info@example.org</nowiki></tt> |mailto:info@example.org |- |mailto unnamed |<tt><nowiki>[mailto:info@example.org]</nowiki></tt> |[mailto:info@example.org] |- |mailto named |<tt><nowiki>[mailto:info@example.org info]</nowiki></tt> |[mailto:info@example.org info] |- |redirect |<tt><nowiki>#REDIRECT [[Main Page]]</nowiki></tt> | &rarr; [[Main Page]] |} == External links == [http://meta.wikimedia.org/wiki/Help:Link Help:Link] [[Category:Help|Links]] 4d86f564dcd99840e0fe142f788d55cb009f62e7 12 1417 1506 1429 2007-08-24T08:40:51Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} To add an article to a category put the following at the end of the page you are editing... <nowiki>[[Category:{Name}]]</nowiki> where {Name} is the name of the category you want to add it to. Any number of category tags may be added to the page - the page will be listed in all of them. You can also specify an additional {Sort} parameter that dictates where the page will appear, alphabetically, within the category. This is achieved by using the following markup: <nowiki>[[Category:{Name}|{Sort}]]</nowiki> So for example, to add this page to the 'Help' category, you would use: <nowiki>[[Category:Help|Categories]]</nowiki> Note that we used 'Categories' as the sort parameter. Without this the page would be listed under 'H' for 'Help:Categories', instead of under 'C', which is more useful. Other situations where you might want to use the sort parameter is when you have articles about people that are titled as <code>FirstName LastName</code> but within the category you want them listed as <code>LastName, FirstName</code>. Another way to sort the article in the correct letter without the namespace is <nowiki>[[Category:Help|{{PAGENAME}}]]</nowiki> This is extremely helpful when using templates which include a category tag. ''Note: the {sort} parameter does '''not''' affect how the page title is displayed within the category listing, just how it is ordered. In the above example, the link to this page will still be 'Help:Categories', and not 'Categories' as you might expect!'' == Linking to Category Pages == To create a link to a category page: <nowiki>[[:Category:{name}]]</nowiki> If you were linking to the Category Page for Help on MediaWiki, the link would look like this: [[:Category:Help]] If you want to display alternate text for the link: <nowiki>[[:Category:{name}|{alternate text}]]</nowiki> Here is an example of the same link to the Category Page for Help on MediaWiki as above, but with alternative text: [[:Category:Help|MediaWiki Help Index]] == Categorize Categories== Categories themselves and other uploaded files like Pictures can be categorized exactly like normal pages. It is useful to connect the article-categories with categories already in place to establish connections and hierarchies. To this end, after saving the article, follow the category links at the end of the page to see, if the category is already in place and if not, categorize them until you connect them with an existing category. ==See also== * [http://meta.wikimedia.org/wiki/Help:Category WikiMedia:Help:Category] [[Category:Help|Categories]] [[Category:Category| ]] 58bc26cf65328aae77774d4aea568a8a4f83afdc 12 1418 1508 1431 2007-08-24T08:40:51Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} __TOC__ {| {{Prettytable}} |- |{{Hl3}}|'''Description''' |{{Hl3}}|'''You type''' |{{Hl3}}|'''You get''' |- |Embed image<br /> (with alt text) | <code><nowiki>[[Image:Example.jpg|Sunflowers]]</nowiki></code> | [[Image:Example.jpg|Sunflowers]] |- |Link to description page | <code><nowiki>[[:Image:Example.jpg]]</nowiki></code><br /> <code><nowiki>[[:Image:Example.jpg|Sunflowers]]</nowiki></code> | [[:Image:Example.jpg]]<br /> [[:Image:Example.jpg|Sunflowers]] |- |Link directly to file | <code><nowiki>[[Media:Example.jpg]]</nowiki></code><br /> <code><nowiki>[[Media:Example.jpg|Sunflowers]]</nowiki></code> | [[Media:Example.jpg]]<br /> [[Media:Example.jpg|Sunflowers]] |- |Thumbnail<br /> (centered, 100 pixels<br /> wide, with caption) | <code><nowiki>[[Image:Example.jpg|center|thumb|100px|Sunflowers]]</nowiki></code> | [[Image:Example.jpg|center|thumb|100px|Sunflowers]] |- |Border<br /> (100 pixels) <br /> Results in a very small gray border <!-- Note: because the fine gray border can not be seen when used on the "Sunflowers"-image an image is used with a withe background --> | <code><nowiki>[[Image:Exampe.jpg|border|100px]]</nowiki></code> | [[Image:Example.jpg|border|100px]]<br> (note that the border is not visible when used with this image) |- |Frameless<br />Like thumbnail, respect user preferences for image width but without border and no right float. | <code><nowiki>[[Image:Example.jpg|frameless]]</nowiki></code> | [[Image:Example.jpg|frameless]] |} ==Uploaded files== To use an image or other file which has been [[Help:Managing files|uploaded to the wiki]], use: *<code><nowiki>[[Image:Example.jpg]]</nowiki></code> If you add a pipe (|) and some text after the filename, the text will be used as alternative text for text-only browsers: *<code><nowiki>[[Image:Example.jpg|alternative text]]</nowiki></code> If you don't want to display the image, you can link to the file's description page, by adding a colon: *<code><nowiki>[[:Image:Example.jpg]]</nowiki></code> To bypass the description page and link directly to a file , use the "Media" pseudo-namespace: *<code><nowiki>[[Media:Example.jpg]]</nowiki></code> You can also make piped links if you want some other text to be used: *<code><nowiki>[[:Image:Example.jpg|link text]]</nowiki></code> *<code><nowiki>[[Media:Example.jpg|link text]]</nowiki></code> ===Advanced options=== The full syntax for displaying an image is: <code><nowiki>[[Image:{name}|{options}]]</nowiki></code> Where options can be zero or more of the following, separated by pipes: *<code>thumb</code>, <code>thumbnail</code>, or <code>frame</code>: Controls how the image is formatted *<code>left</code>, <code>right</code>, <code>center</code>, <code>none</code>: Controls the alignment of the image on the page *<code>{width}px</code>: Resizes the image to the given width in pixels *<code>{caption text}</code> The options can be given in any order. If a given option does not match any of the other possibilities, it is assumed to be the caption text. Caption text can contain wiki links or other formatting. ===Gallery of images=== It's easy to make a gallery of thumbnails with the <code><nowiki><gallery></nowiki></code> tag. The syntax is: <pre> <gallery> Image:{filename}|{caption} Image:{filename}|{caption} {...} </gallery> </pre> Captions are optional, and may contain wiki links or other formatting. for example: <pre> <gallery> Image:Example.jpg|Item 1 Image:Example.jpg|a link to [[Help:Contents]] Image:Example.jpg Image:Example.jpg Image:Example.jpg|''italic caption'' </gallery> </pre> is formatted as: <gallery> Image:Example.jpg|Item 1 Image:Example.jpg|a link to [[Help:Contents]] Image:Example.jpg Image:Example.jpg Image:Example.jpg|''italic caption'' </gallery> ==Other files== You can link to an external file using the same syntax used for linking to an external web page. *<code><nowiki>[http://url.for/some/image.png]</nowiki></code> Or with different text: *<code><nowiki>[http://url.for/some/image.png link text here]</nowiki></code> If it is enabled on your wiki (see [http://www.mediawiki.org/wiki/Manual:$wgAllowExternalImages Manual:$wgAllowExternalImages]), you can also embed external images. To do that, simply insert the image's url: *<code><nowiki>http://url.for/some/image.png</nowiki></code> == Administration of images/files == See here: [http://www.mediawiki.org/wiki/Manual:Image_Administration Manual:Image Administration] ==External Links== [http://meta.wikimedia.org/wiki/Help:Images_and_other_uploaded_files Images and other uploaded files] [[Category:Help|Images]] cf25ea60b9a82f25443430313cc49ec8df4c4c1b 12 1419 1510 1486 2007-08-24T08:40:51Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} If you have standard texts you want to include on several pages, the MediaWiki template feature comes into play (like the tag above which is included in the pages of the Help namespace). ==Creating a template== Template names are prefixed with <code>Template:</code>, you create them like any other wiki page. ==Using a template== Templates are wiki pages which can be used in other pages in three ways: *<code><nowiki>{{Name}}</nowiki></code> includes the content of the template at the moment the page containing it is fetched <nowiki>"[[Template:Name]]"</nowiki>. *<code><nowiki>{{subst:Name}}</nowiki></code> inserts the content of the template into the code of the page in a form that is editable normally <nowiki>"[[Template:Name]]"</nowiki> after you have saved your text. *<code><nowiki>{{msgnw:Name}}</nowiki></code> when the page containing it is fetched, includes the template in a form that displays it as raw wiki syntax, like <code><nowiki>&lt;nowiki&gt;</nowiki></code> does ==Using parameters in templates== <div style="float:right"> {| {{Prettytable}} |- |{{Hl2}} colspan="2" align="center" |'''Template with numbered parameters''' |- | colspan="2" | <pre><nowiki> '''A little thank you...'''<br> <small>for {{{1}}}.<br> hugs, {{{2}}}</small> </nowiki></pre> |- |{{Hl2}}|'''You type''' |{{Hl2}}|'''You get''' |- |<code><nowiki>{{Thankyou|all|Joe}}</nowiki></code> | {{Thankyou|all|Joe}} |- |{{Hl2}} colspan="2" align="center" |'''with named parameters''' |- | colspan="2" | <pre><nowiki> '''A little thank you...'''<br> <small>for {{{reason}}}.<br> hugs, {{{signature}}}</small> </nowiki></pre> |- |{{Hl2}}|'''You type''' |{{Hl2}}|'''You get''' |- |<pre><nowiki>{{Thankyou |reason=all |signature=Joe}}</nowiki></pre> | {{Thankyou|all|Joe}} |} </div> You can define parameters in templates either numbered as <code><nowiki>{{{1}}}</nowiki></code> or named <code><nowiki>{{{param}}}</nowiki></code>. '''Example:''' You want a little thank you note you can put on the talk page of other users. It will contain a reason and your signature. You could create [[Template:Thankyou]] to enter your text, as in the example in the table. When using the template on a page, you fill in the parameter values, separated by a pipe char (|): <code><nowiki>{{Thankyou|all|Joe}}</nowiki></code>. For named parameters use "name=value" pairs separated by a pipe char: <code><nowiki>{{Thankyou|reason=all|signature=Joe}}</nowiki></code>. The advantage of using named parameters in your template is that they are flexible in order. It also makes the template easier to understand if you have many parameters. If you want to change the order of numbered parameters, you have to mention them explicitly: <code><nowiki>{{Thankyou|2=Joe|1=all}}</nowiki></code>. You can also provide default values for parameters, i.e. values that are going to be used if no value is provided for a parameter. For example, <code><nowiki>{{{reason|all}}}</nowiki></code> would result in ''"all"'' if no value was provided for the parameter <tt>reason</tt>. ==Control template inclusion== You can control template inclusion by the use of <code><nowiki><noinclude></nowiki></code> and <code><nowiki><includeonly></nowiki></code> tags. Anything between <code><nowiki><noinclude></nowiki></code> and <code><nowiki></noinclude></nowiki></code> will be processed and displayed only when the page is being viewed directly, not included. Possible applications are: * Categorising templates * Interlanguage links to similar templates in other languages * Explanatory text about how to use the template The converse is <code><nowiki><includeonly></nowiki></code>. Text between <code><nowiki><includeonly></nowiki></code> and <code><nowiki></includeonly></nowiki></code> will be processed and displayed only when the page is being included. The obvious application is to add all pages containing a given template to a category. Note that the usual update problems apply -- if you change the categories inside a template, the categories of the referring pages won't be updated until those pages are edited. ==Organizing templates== For templates to be effective users need to find them and be able to use them. A simple technique is to include an example on the template page. For example: <div style="display:table; width:auto;"><pre> <noinclude> ==Usage== Allows to establish a link to a subject: {{NameOfTemplate|Term1+Term2+Term3}} </noinclude> </pre></div> Then, an editor can simply copy and paste the example to create a similar page. ==See Also== *[[Help:Variables]] -- fancy stuff you may find in some templates ==External links== [http://meta.wikimedia.org/wiki/Help:Template Help:Template] [[Category:Help|{{PAGENAME}}]] ad8e8a7fa096c0ae3a2dece6779d12daf7029617 12 1420 1512 1435 2007-08-24T08:40:51Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Tables may be authored in wiki pages using either HTML table elements directly, or using wikicode formatting to define the table. HTML table elements and their use are well described on various web pages and will not be discussed here. The benefit of wikicode is that the table is constructed of character symbols which tend to make it easier to perceive the table structure in the article editing view compared to HTML table elements. A good general rule of thumb is to avoid using a table unless you need to. Table markup often complicates page editing. == Wiki table markup summary == {|cellpadding="5" cellspacing="0" border="1" width="600" |<nowiki>{|</nowiki> || start '''table''' |- |<nowiki>|+</nowiki> || table '''caption,''' ''optional;'' only one per table and between table start and first row |- |<nowiki>|-</nowiki> || '''table row,''' ''optional on first row'' -- wiki engine assumes the first row |- |<nowiki>!</nowiki> || '''table header''' cell, ''optional.'' Consecutive table headers may be added on same line separated by double marks (!!) or start on new lines, each with its own single mark (!). |- |<nowiki>|</nowiki> || '''table data''' cell, ''required!'' Consecutive table data cells may be added on same line separated by double marks (<nowiki>||</nowiki>) or start on new lines, each with its own single mark (<nowiki>|</nowiki>). |- |<nowiki>|}</nowiki> || end '''table''' |} *The above marks must '''start on a new line''' except the double || and !! for optionally adding consecutive cells to a line. *'''XHTML attributes.''' Each mark, except table end, optionally accepts one or more XHTML attributes. Attributes must be on the same line as the mark. Separate attributes from each other with a single space. **Cells and caption (<nowiki>| or ||, ! or !!, and |+</nowiki>) hold content. So separate any attributes from content with a single pipe (|). Cell content may follow on same line or on following lines. **Table and row marks (<nowiki>{| and |-</nowiki>) do not directly hold content. Do ''not'' add pipe (|) after their optional attributes. If you erroneously add a pipe after attributes for the table mark or row mark the parser will delete it ''and'' your final attribute if it was touching the erroneous pipe! *'''Content''' may (a) follow its cell mark on the same line after any optional XHTML attributes or (b) on lines below the cell mark. Content that uses wiki markup that itself needs to start on a new line, such as lists, headers, or nested tables, must of course be on its own new line. ==Simple table== ===Plain=== The following table lacks borders and good spacing but shows the simplest wiki markup table structure {| width="100%" |width="50%"| {| |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ===Alternative=== For more table-ish looking wiki markup cells can be listed on one line separated by ||. This does not scale well for longer cell content such as paragraphs. It works well for short bits of content however, such as our example table. Extra spaces within cells in the wiki markup can be added, as I have done in the wiki markup below, to make the wiki markup itself look better but they do not affect the actual table rendering. HTML attributes can be added to this table following the examples in other tables on this page but have been left out of the following example for simplicity. {| width="100%" |width="50%"| {| | Orange || Apple || more |- | Bread || Pie || more |- | Butter || Ice cream || and more |} |width="50%"| <pre> {| | Orange || Apple || more |- | Bread || Pie || more |- | Butter || Ice cream || and more |} </pre> |} ===With HTML attributes=== You can add HTML attributes to make your table look better ====border="1"==== {| width="100%" |width="50%"| {| border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ====align="center" border="1"==== {| width="100%" |width="50%"| {| align="center" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| align="center" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ====align="right" border="1"==== You can put attributes on individual '''cells.''' Numbers for example may look better aligned right {| width="100%" |width="50%"| {| border="1" |Orange |Apple |align="right"|12,333.00 |- |Bread |Pie |align="right"|500.00 |- |Butter |Ice cream |align="right"|1.00 |} |width="50%"| <pre> {| border="1" |Orange |Apple |align="right"|12,333.00 |- |Bread |Pie |align="right"|500.00 |- |Butter |Ice cream |align="right"|1.00 |} </pre> |} You can put attributes on individual '''rows,''' too. {| width="100%" |width="50%"| {| border="1" |Orange |Apple |align="right"|12,333.00 |- |Bread |Pie |align="right"|500.00 |- style="font-style:italic;color:green;" |Butter |Ice cream |align="right"|1.00 |} |width="50%"| <pre> {| border="1" |Orange |Apple |align="right"|12,333.00 |- |Bread |Pie |align="right"|500.00 |- style="font-style:italic;color:green;" |Butter |Ice cream |align="right"|1.00 |} </pre> |} ====cellspacing="0" border="1"==== {| width="100%" |width="50%"| {| cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ====cellpadding="20" cellspacing="0" border="1"==== {| width="100%" |width="50%"| {| cellpadding="20" cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| cellpadding="20" cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ===With HTML attributes and CSS styles=== CSS style attributes can be added with or without other HTML attributes ====style="color:green;background-color:#ffffcc;" cellpadding="20" cellspacing="0" border="1"==== {| width="100%" |width="50%"| {| style="color:green;background-color:#ffffcc;" cellpadding="20" cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| style="color:green;background-color:#ffffcc;" cellpadding="20" cellspacing="0" border="1" |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ==Table with TH headings== TH (HTML table headings) can be created by using ! instead of |. Headings usually show up bold and centered by default. ===Top headings=== ====Each column==== {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" !Yummy !Yummier |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" !Yummy !Yummier |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ====Colspan="2"==== {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" !colspan="2"|Yummies |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" ! colspan="2"|Yummies |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ===Side headings=== ====Default==== {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" !Fruit |Orange |Apple |- !Dish |Bread |Pie |- !Complement |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" !Fruit |Orange |Apple |- !Dish |Bread |Pie |- !Complement |Butter |Ice cream |} </pre> |} ====Right justify==== Right justified side headings can be done as follows {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" !align="right" |Fruit |Orange |Apple |- !align="right" |Dish |Bread |Pie |- !align="right" |Complement |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" !align="right" |Fruit |Orange |Apple |- !align="right" |Dish |Bread |Pie |- !align="right" |Complement |Butter |Ice cream |} </pre> |} ==Caption== A '''table caption''' can be added to the top of any table as follows {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" |+Food complements |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" |+Food complements |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} '''Attributes''' can be added to the caption as follows {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" |+align="bottom" style="color:#e76700;"|''Food complements'' |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" |+align="bottom" style="color:#e76700;"|''Food complements'' |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ==Table with H1, H2, H3 etc. headings== HTML H1, H2, H3, H4 etc. headings can be created the standard wiki markup way with ==equal== signs and '''must be on a line all by themselves''' to work. '''Preview the whole table.''' If you click on an edit tab for a heading ''within'' a table, edit, and preview, the parent table will display erroneously broken because part of it will be missing. Keep the heading hierarchy consistent with the rest of the page so that the table of contents at page top works correctly. {| width="100%" |width="50%"| {| border="1" cellpadding="20" cellspacing="0" |colspan="2"| ===Yummiest=== |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} |width="50%"| <pre> {| border="1" cellpadding="20" cellspacing="0" |colspan="2"| ===Yummiest=== |- |Orange |Apple |- |Bread |Pie |- |Butter |Ice cream |} </pre> |} ==Caveat== ===Negative numbers=== Negative value minus sign can break your table (it may display missing some values) if you start a cell on a new line with a negative number or a parameter that evaluates to a negative number (|-6) because that is the wiki markup for table row, not table cell. To avoid this, insert a space before the value (| -6) or use in-line cell markup (||-6). [[Category:Help|Tables]] [[Category:Tables]] 219e8f346e0cd7ca043e36b3bc5de04cefd5b9af 12 1421 1514 1437 2007-08-24T08:40:51Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} MediaWiki knows some defined variables which can be used in texts. They are rendered differently depending on the time, the site and the page. Their syntax is similar to templates. If you want a page to show always the current time, you use <code><nowiki>{{CURRENTTIME}}</nowiki></code>. To distinguish variables from templates, most variables are capitalized. <center> {| {{Prettytable}} |- !{{Hl3}}|'''Description''' !{{Hl3}}|'''You type''' !{{Hl3}}|'''You get''' |- ! {{Hl2}} colspan="3" | '''Date & Wiki variables''' |- | Current month |<nowiki>{{CURRENTMONTH}}</nowiki> |{{CURRENTMONTH}} |- | Current month |<nowiki>{{CURRENTMONTHNAME}}</nowiki> |{{CURRENTMONTHNAME}} |- | Genitive form<br>of current month name |<nowiki>{{CURRENTMONTHNAMEGEN}}</nowiki> |{{CURRENTMONTHNAMEGEN}} |- | Abbreviated month name |<nowiki>{{CURRENTMONTHABBREV}}</nowiki> |{{CURRENTMONTHABBREV}} |- |Day of the month |<nowiki>{{CURRENTDAY}}</nowiki> |{{CURRENTDAY}} |- |Day of the week |<nowiki>{{CURRENTDOW}}</nowiki> |{{CURRENTDOW}} |- |Day of the week |<nowiki>{{CURRENTDAYNAME}}</nowiki> |{{CURRENTDAYNAME}} |- | Week |<nowiki>{{CURRENTWEEK}}</nowiki> |{{CURRENTWEEK}} |- | Year |<nowiki>{{CURRENTYEAR}}</nowiki> |{{CURRENTYEAR}} |- | current time |<nowiki>{{CURRENTTIME}}</nowiki> |{{CURRENTTIME}} |- | number of pages in this Wiki (total) |<nowiki>{{NUMBEROFPAGES}}</nowiki> |{{NUMBEROFPAGES}} |- | numbers of articles in this Wiki |<nowiki>{{NUMBEROFARTICLES}}</nowiki> |{{NUMBEROFARTICLES}} |- | numbers of uploaded files |<nowiki>{{NUMBEROFFILES}}</nowiki> |{{NUMBEROFFILES}} |- ! {{Hl2}} colspan="3" | '''Namespaces''' |- | |<nowiki>{{ns:-2}} or {{ns:Media}}</nowiki> |{{ns:-2}} |- | |<nowiki>{{ns:-1}} or {{ns:Special}}</nowiki> |{{ns:-1}} |- | |<nowiki>{{ns:1}} or {{ns:Talk}}</nowiki> | {{ns:1}} |- | | <nowiki>{{ns:2}} or {{ns:User}}</nowiki> | {{ns:2}} |- | | <nowiki>{{ns:3}} or {{ns:User_talk}}</nowiki> | {{ns:3}} |- | | <nowiki>{{ns:4}} or {{ns:Project}}</nowiki> | {{ns:4}} |- | | <nowiki>{{ns:5}} or {{ns:Project_talk}}</nowiki> | {{ns:5}} |- | | <nowiki>{{ns:6}} or {{ns:Image}}</nowiki> | {{ns:6}} |- | | <nowiki>{{ns:7}} or {{ns:Image_talk}}</nowiki> | {{ns:7}} |- | | <nowiki>{{ns:8}} or {{ns:MediaWiki}}</nowiki> | {{ns:8}} |- | | <nowiki>{{ns:9}} or {{ns:MediaWiki_talk}}</nowiki> | {{ns:9}} |- | | <nowiki>{{ns:10}} or {{ns:Template}}</nowiki> | {{ns:10}} |- | | <nowiki>{{ns:11}} or {{ns:Template_talk}}</nowiki> | {{ns:11}} |- | | <nowiki>{{ns:12}} or {{ns:Help}}</nowiki> | {{ns:12}} |- | | <nowiki>{{ns:13}} or {{ns:Help_talk}}</nowiki> | {{ns:13}} |- | | <nowiki>{{ns:14}} or {{ns:Category}}</nowiki> | {{ns:14}} |- | | <nowiki>{{ns:15}} or {{ns:Category_talk}}</nowiki> | {{ns:15}} |- ! {{Hl2}} colspan="3" | '''Server specific''' |- | Sitename | <nowiki>{{SITENAME}}</nowiki> | {{SITENAME}} |- | Servername | <nowiki>{{SERVERNAME}}</nowiki> | {{SERVERNAME}} |- | Server | <nowiki>{{SERVER}}</nowiki> | {{SERVER}} |- |- | Script path | <nowiki>{{SCRIPTPATH}}</nowiki> | {{SCRIPTPATH}} |- | Local part of the URL<br> | <nowiki>{{localurl:pagename}}</nowiki> | {{localurl:pagename}} |- | | <nowiki>{{localurl:pagename|</nowiki>''query string''<nowiki>}}</nowiki> | {{localurl:pagename|''query string''}} |- ! {{Hl2}} colspan="3" | '''Page related''' |- | Pagename | <nowiki>{{PAGENAME}}</nowiki> | {{PAGENAME}} |- | Pagename<br>(URL encoded) | <nowiki>{{PAGENAMEE}}</nowiki> | {{PAGENAMEE}} |- | Namespace | <nowiki>{{NAMESPACE}}</nowiki> | {{NAMESPACE}} |- | Revision ID | <nowiki>{{REVISIONID}}</nowiki> | {{REVISIONID}} |} </center> ==External links== [http://meta.wikimedia.org/wiki/Help:Variable Help:Variable] [[Category:Help|Variables]] 6359ec43c64fd4526119002a92e3f325057850fa 12 1422 1516 1439 2007-08-24T08:40:51Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} ==Upload a file== # In the [[Help:Navigation#Sidebar|sidebar]], under '''toolbox''', click "Upload file" # Next to the '''Source filename''', click '''Browse''' to locate the file on your computer. # Change the '''Destination filename''' to something descriptive, if necessary. # Fill in the '''Summary''', if necessary. # Click the '''Upload file''' button. After you've uploaded some files, [[Help:Images|use them in your articles]]. [[Category:Help|Managing files]] 4c43300bcaf807c82bf3164f90ebbbb7da883fdc 12 1423 1518 1517 2007-08-24T08:40:51Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Clicking on the [[special:preferences|my preferences]] link in the upper right while logged in allows you to change your preferences. ==Date format== The following is normally rendered depending on preferences: <pre> [[2001-01-05]] (or [[2001]]-[[01-05]]) (with leading zeros) [[2001]] [[January 5]] ([[2001]] [[January 05]]) [[January 5]], [[2001]] ([[January 05]], [[2001]]) [[5 January]] [[2001]] ([[05 January]] [[2001]]) [[January 5]] ([[January 05]]) [[5 January]] ([[05 January]]) </pre> With your current preference setting on this project the seven are rendered as follows: #[[2001-01-05]] ([[2001]]-[[01-05]]) #[[2001]] [[January 5]] ([[2001]] [[January 05]]) #[[January 5]], [[2001]] ([[January 05]], [[2001]]) #[[5 January]] [[2001]] ([[05 January]] [[2001]]) #[[January 5]] ([[January 05]]) #[[5 January]] ([[05 January]]) The user-specified date format does not seem to work on the mw: wiki for links! It works in Recent Changes etc. == See also == * [[Help:Skins]] ==External Links== [http://meta.wikimedia.org/wiki/Help:Preferences Help:Preferences] [[Category:Help|Preferences]] a27548c08b7cba5173405bcb2a16b8c2dfbd0479 12 1424 1520 1443 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Clicking on the [[special:preferences|my preferences]] link in the upper right while logged in then click on the '''Skin''' button to change your skin. You can also preview the skin by clicking the '''(preview)''' links next to each skin. You can make changes to the current skin's stylesheet file (CSS) by creating a subpage of your userpage, "[[Special:Mypage/monobook.css|User:Yourname/monobook.css]]" for example. This requires your site admin to have enabled this feature -- if it is, you will see advice text at the top of your custom CSS page about clearing your browser's cache. == See also == * [[Help:Preferences]] == External Links == [http://meta.wikimedia.org/wiki/Skins Skins] [[Category:Help]] bdd8b61a54312b596a9613de1cc200211f8a599b 12 1425 1522 1445 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} Articles without a ':' in their name are in the main namespace. This namespace is for articles; documents that aren't directly related to this may go in other namespaces. When articles are in another namespace, they are specified as ''namespace'':''article''; i.e. this page is ''{{NAMESPACE}}'':''{{PAGENAME}}''. ;Help: Documentation about working with the wiki software. This could be mirrored from outside sites, or locally written. ;Image: For descriptions of uploaded files. You shouldn't create these directly; they are created when you click the [[special:upload|Upload file]] link in the toolbox. ;Media: Use this namespace to link to uploaded files directly, rather than through the description pages. ;MediaWiki: Use this namespace to change the default system messages, [http://meta.wikimedia.org/wiki/Help:System message See Help:System message on meta]. ;Project: Information about this wiki; i.e. policies that apply here. This namespace also has an alias, which is the name of the wiki installation. ;Talk: Each page has a corresponding ''discussion'' page. This can be used for feedback/comments about that page, or other local notes that another group may want to associate with the page, without modifying the document directly, or for any other additional information to associate with the document. ;Template: This is used for meta-information that is to be transcluded into multiple documents, such as tags to mark the status of a document. ;User: For personal notes. Each User has a corresponding user page for their own information. Users can also create subpages, by using a / after their name. ;User_talk: The discussion page on a user's page can be used for leaving messages. If this page is edited, the next time that user logs in they will see a box notifying them that they have new messages [[Category:Help|Namespaces]] ce056b03e0db36e8d30fd6d10d45a2fa01a85210 12 1426 1524 1447 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} '''Special pages''' are pages that are created by the software on demand. They are located in their own [[Help:Namespaces|namespace]] ''Special:'' and are not editable directly as other pages. Some special pages depend on the [[Help:Preferences|preferences]] that have been set by a user, e.g. the number of titles which is displayed on a user's watchlist. ==List of special pages== Clicking the link ''[[Special:Specialpages]]'' will take you to a list of all special pages on a wiki. ==External links== [http://meta.wikimedia.org/wiki/Help:Special_page Help:Special page] [[Category:Help|Special pages]] [[Category:Special Pages| ]] 6d62eb7b20ac98315b2a1e21839b0d1c87ddabdb 12 1427 1526 1449 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki '''Range blocks''' are technical restrictions applied through [[Special:Blockip]] to a group of [http://en.wikipedia.org/wiki/IP_addresses IP addresses] that prevent them from editing, creating new accounts, sending email through the wiki interface, et cetera. Registered accounts editing from these IP addresses will also be blocked, unless you check the box to only block anonymous editors. Range blocking is enabled on all [http://en.wikipedia.org/wiki/Wikimedia_Foundation Wikimedia] wikis; to enable it on other wikis, add "<code><nowiki>$wgSysopRangeBans = true;</nowiki></code>" in [http://www.mediawiki.org/wiki/LocalSettings.php LocalSettings.php]. ==How to block ranges== To block an IP range from [[Special:Blockip]], enter the first IP address in the range followed by a forward slash and a [http://en.wikipedia.org/wiki/Classless_inter-domain_routing Classless inter-domain routing] (CIDR) suffix. '''You should avoid performing range blocks unless you understand what you are doing, or potentially blocking tens of thousands of extra IP addresses is not a problem.''' ===Technical explanation=== CIDR notation is written as the IP address, a slash, and the CIDR affix (for example, the IPv4 "<code>10.2.3.41/24</code>" or IPv6 "<code>a3:b:c1:d:e:f:1:21/24</code>"). The CIDR affix is the number of starting digits every IP address in the range have in common when written in binary. For example: "<code>10.10.1.32</code>" is binary "<code>00001010.00001010.00000001.00100000</code>", so <code>10.10.1.32/27</code> will match the first 27 digits ("<code><u>00001010</u>.<u>00001010</u>.<u>00000001</u>.<u>001</u>00000</code>"). The IP addresses <code>10.10.1.32</code>–<code>10.10.1.63</code>, when converted to binary, all have the same 27 first digits and will be blocked if <code>10.10.1.32/27</code> is blocked. As the CIDR affix increases, the block affects less IP addresses (see [[#Table|table of example ranges]]). CIDR affixes are not the same for IPv4 addresses as they are for IPv6 addresses. ===Calculating the CIDR affix=== You can use the [[#Table|table of sample ranges]] below to guess the range, or manually calculate the range: # Convert the first and last IP addresses in the range to binary using [[#conversion_list|the list below]]. # Place both IP addresses one atop the other, and count how many starting digits are exactly alike. This is the CIDR affix. # Double-check! Being off by one digit could extend your block by thousands of addresses. The example below calculates the CIDR range between 69.208.0.0 and 69.208.0.255. Note that this is a simple example; some groups of IP addresses do not so neatly fit CIDR affixes, and need multiple different-sized blocks to block the exact range. IP addresses: 69.208.0.0 69.208.0.255 &nbsp; Convert to binary: 0100 0101.1101 0000.0000 0000.0000 0000 0100 0101.1101 0000.0000 0000.1111 1111 &nbsp; Count identical first numbers: '''0100 0101.1101 0000.0000 0000'''.0000 0000 '''0100 0101.1101 0000.0000 0000'''.1111 1111 |____________________________| 24 digits &nbsp; CIDR range: 69.208.0.0/24 </pre> ===<span id="conversion list">List of decimal->binary conversions</span>=== <div style="-moz-column-count:4; -moz-column-gap:2em; column-count:4; column-gap:2em; font-size:0.9em; font-family:monospace;"> Decimal -> Binary 0 -> 0000 0000 1 -> 0000 0001 2 -> 0000 0010 3 -> 0000 0011 4 -> 0000 0100 5 -> 0000 0101 6 -> 0000 0110 7 -> 0000 0111 8 -> 0000 1000 9 -> 0000 1001 10 -> 0000 1010 11 -> 0000 1011 12 -> 0000 1100 13 -> 0000 1101 14 -> 0000 1110 15 -> 0000 1111 16 -> 0001 0000 17 -> 0001 0001 18 -> 0001 0010 19 -> 0001 0011 20 -> 0001 0100 21 -> 0001 0101 22 -> 0001 0110 23 -> 0001 0111 24 -> 0001 1000 25 -> 0001 1001 26 -> 0001 1010 27 -> 0001 1011 28 -> 0001 1100 29 -> 0001 1101 30 -> 0001 1110 31 -> 0001 1111 32 -> 0010 0000 33 -> 0010 0001 34 -> 0010 0010 35 -> 0010 0011 36 -> 0010 0100 37 -> 0010 0101 38 -> 0010 0110 39 -> 0010 0111 40 -> 0010 1000 41 -> 0010 1001 42 -> 0010 1010 43 -> 0010 1011 44 -> 0010 1100 45 -> 0010 1101 46 -> 0010 1110 47 -> 0010 1111 48 -> 0011 0000 49 -> 0011 0001 50 -> 0011 0010 51 -> 0011 0011 52 -> 0011 0100 53 -> 0011 0101 54 -> 0011 0110 55 -> 0011 0111 56 -> 0011 1000 57 -> 0011 1001 58 -> 0011 1010 59 -> 0011 1011 60 -> 0011 1100 61 -> 0011 1101 62 -> 0011 1110 63 -> 0011 1111 64 -> 0100 0000 65 -> 0100 0001 66 -> 0100 0010 67 -> 0100 0011 68 -> 0100 0100 69 -> 0100 0101 70 -> 0100 0110 71 -> 0100 0111 72 -> 0100 1000 73 -> 0100 1001 74 -> 0100 1010 75 -> 0100 1011 76 -> 0100 1100 77 -> 0100 1101 78 -> 0100 1110 79 -> 0100 1111 80 -> 0101 0000 81 -> 0101 0001 82 -> 0101 0010 83 -> 0101 0011 84 -> 0101 0100 85 -> 0101 0101 86 -> 0101 0110 87 -> 0101 0111 88 -> 0101 1000 89 -> 0101 1001 90 -> 0101 1010 91 -> 0101 1011 92 -> 0101 1100 93 -> 0101 1101 94 -> 0101 1110 95 -> 0101 1111 96 -> 0110 0000 97 -> 0110 0001 98 -> 0110 0010 99 -> 0110 0011 100 -> 0110 0100 101 -> 0110 0101 102 -> 0110 0110 103 -> 0110 0111 104 -> 0110 1000 105 -> 0110 1001 106 -> 0110 1010 107 -> 0110 1011 108 -> 0110 1100 109 -> 0010 1101 110 -> 0110 1110 111 -> 0110 1111 112 -> 0111 0000 113 -> 0111 0001 114 -> 0111 0010 115 -> 0111 0011 116 -> 0111 0100 117 -> 0111 0101 118 -> 0111 0110 119 -> 0111 0111 120 -> 0111 1000 121 -> 0111 1001 122 -> 0111 1010 123 -> 0111 1011 124 -> 0111 1100 125 -> 0111 1101 126 -> 0111 1110 127 -> 0111 1111 128 -> 1000 0000 129 -> 1000 0001 130 -> 1000 0010 131 -> 1000 0011 132 -> 1000 0100 133 -> 1000 0101 134 -> 1000 0110 135 -> 1000 0111 136 -> 1000 1000 137 -> 1000 1001 138 -> 1000 1010 139 -> 1000 1011 140 -> 1000 1100 141 -> 1000 1101 142 -> 1000 1110 143 -> 1000 1111 144 -> 1001 0000 145 -> 1001 0001 146 -> 1001 0010 147 -> 1001 0011 148 -> 1001 0100 149 -> 1001 0101 150 -> 1001 0110 151 -> 1001 0111 152 -> 1001 1000 153 -> 1001 1001 154 -> 1001 1010 155 -> 1001 1011 156 -> 1001 1100 157 -> 1001 1101 158 -> 1001 1110 159 -> 1001 1111 160 -> 1010 0000 161 -> 1010 0001 162 -> 1010 0010 163 -> 1010 0011 164 -> 1010 0100 165 -> 1010 0101 166 -> 1010 0110 167 -> 1010 0111 168 -> 1010 1000 169 -> 1010 1001 170 -> 1010 1010 171 -> 1010 1011 172 -> 1010 1100 173 -> 1010 1101 174 -> 1010 1110 175 -> 1010 1111 176 -> 1011 0000 177 -> 1011 0001 178 -> 1011 0010 179 -> 1011 0011 180 -> 1011 0100 181 -> 1011 0101 182 -> 1011 0110 183 -> 1011 0111 184 -> 1011 1000 185 -> 1011 1001 186 -> 1011 1010 187 -> 1011 1011 188 -> 1011 1100 189 -> 1011 1101 190 -> 1011 1110 191 -> 1011 1111 192 -> 1100 0000 193 -> 1100 0001 194 -> 1100 0010 195 -> 1100 0011 196 -> 1100 0100 197 -> 1100 0101 198 -> 1100 0110 199 -> 1100 0111 200 -> 1100 1000 201 -> 1100 1001 202 -> 1100 1010 203 -> 1100 1011 204 -> 1100 1100 205 -> 1100 1101 206 -> 1100 1110 207 -> 1100 1111 208 -> 1101 0000 209 -> 1101 0001 210 -> 1101 0010 211 -> 1101 0011 212 -> 1101 0100 213 -> 1101 0101 214 -> 1101 0110 215 -> 1101 0111 216 -> 1101 1000 217 -> 1101 1001 218 -> 1101 1010 219 -> 1101 1011 220 -> 1101 1100 221 -> 1101 1101 222 -> 1101 1110 223 -> 1101 1111 224 -> 1110 0000 225 -> 1110 0001 226 -> 1110 0010 227 -> 1110 0011 228 -> 1110 0100 229 -> 1110 0101 230 -> 1110 0110 231 -> 1110 0111 232 -> 1110 1000 233 -> 1110 1001 234 -> 1110 1010 235 -> 1110 1011 236 -> 1110 1100 237 -> 1010 1101 238 -> 1110 1110 239 -> 1110 1111 240 -> 1111 0000 241 -> 1111 0001 242 -> 1111 0010 243 -> 1111 0011 244 -> 1111 0100 245 -> 1111 0101 246 -> 1111 0110 247 -> 1111 0111 248 -> 1111 1000 249 -> 1111 1001 250 -> 1111 1010 251 -> 1111 1011 252 -> 1111 1100 253 -> 1111 1101 254 -> 1111 1110 255 -> 1111 1111 </div> ===<span id="Table">Table of sample ranges</span>=== The table below shows the IP blocks each CIDR suffix affects. Note that MediaWiki only supports blocking CIDR suffixes 16&ndash;32. {| class="prettytable" ! CIDR ! Start Range ! End Range ! Total addresses ! Bits selected in IP address |- style="color:gray;" | 69.208.0.0'''/0''' | 0.0.0.0 | 255.255.255.255 | 4,294,967,296 | ********.********.********.******** |- style="color:gray;" | 69.208.0.0'''/1''' | 0.0.0.0 | 127.255.255.255 | 2,147,483,648 | 0*******.********.********.******** |- style="color:gray;" | 69.208.0.0'''/4''' | 65.0.0.0 | 79.255.255.255 | 268,435,456 | 0100****.********.********.******** |- style="color:gray;" | 69.208.0.0'''/8''' | 69.0.0.0 | 69.255.255.255 | 67,108,864 | 01000101.********.********.******** |- style="color:gray;" | 69.208.0.0'''/11''' | 69.208.0.0 | 69.238.255.255 | 2,197,152 | 01000101.110*****.********.******** |- style="color:gray;" | 69.208.0.0'''/12''' | 69.208.0.0 | 69.223.255.255 | 1,048,576 | 01000101.1101****.********.******** |- style="color:gray;" | 69.208.0.0'''/13''' | 69.208.0.0 | 69.215.255.255 | 524,288 | 01000101.11010***.********.******** |- style="color:gray;" | 69.208.0.0'''/14''' | 69.208.0.0 | 69.211.255.255 | 262,144 | 01000101.110100**.********.******** |- style="color:gray;" | 69.208.0.0'''/15''' | 69.208.0.0 | 69.209.255.255 | 131,072 | 01000101.1101000*.********.******** |- | 69.208.0.0'''/16''' | 69.208.0.0 | 69.208.255.255 | 65,536 | 01000101.11010000.********.******** |- | 69.208.0.0'''/17''' | 69.208.0.0 | 69.208.127.255 | 32,768 | 01000101.11010000.0*******.******** |- | 69.208.0.0'''/18''' | 69.208.0.0 | 69.208.63.255 | 16,384 | 01000101.11010000.00******.******** |- | 69.208.0.0'''/19''' | 69.208.0.0 | 69.208.31.255 | 8,192 | 01000101.11010000.000*****.******** |- | 69.208.0.0'''/20''' | 69.208.0.0 | 69.208.15.255 | 4,096 | 01000101.11010000.0000****.******** |- | 69.208.0.0'''/21''' | 69.208.0.0 | 69.208.7.255 | 2,048 | 01000101.11010000.00000***.******** |- | 69.208.0.0'''/22''' | 69.208.0.0 | 69.208.3.255 | 1,024 | 01000101.11010000.000000**.******** |- | 69.208.0.0'''/23''' | 69.208.0.0 | 69.208.1.255 | 512 | 01000101.11010000.0000000*.******** |- | 69.208.0.0'''/24''' | 69.208.0.0 | 69.208.0.255 | 256 | 01000101.11010000.00000000.******** |- | 69.208.0.0'''/25''' | 69.208.0.0 | 69.208.0.127 | 128 | 01000101.11010000.00000000.0******* |- | 69.208.0.0'''/26''' | 69.208.0.0 | 69.208.0.63 | 64 | 01000101.11010000.00000000.00****** |- | 69.208.0.0'''/27''' | 69.208.0.0 | 69.208.0.31 | 32 | 01000101.11010000.00000000.000***** |- | 69.208.0.0'''/28''' | 69.208.0.0 | 69.208.0.15 | 16 | 01000101.11010000.00000000.0000**** |- | 69.208.0.0'''/29''' | 69.208.0.0 | 69.208.0.7 | 8 | 01000101.11010000.00000000.00000*** |- | 69.208.0.0'''/30''' | 69.208.0.0 | 69.208.0.3 | 4 | 01000101.11010000.00000000.000000** |- | 69.208.0.0'''/31''' | 69.208.0.0 | 69.208.0.1 | 2 | 01000101.11010000.00000000.0000000* |- | 69.208.0.0'''/32''' | 69.208.0.0 | 69.208.0.0 | 1 | 01000101.11010000.00000000.00000000 |} ==References== * [http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing Classless Inter-Domain Routing] * [http://www.ccci.com/tools/subcalc/binary.html Converting IP addresses to binary] ==External links== * [http://apps.csc.fi/laskin2.html Netmask calculator] which helps in making the correct decision for range blocks. {{PD Help Page}} [[Category:Help|Range blocks]] dd441b70a5ca16be4ed43f1c05674eb02e0b3143 12 1428 1528 1451 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} [http://meta.wikimedia.org/wiki/Help:User_levels Please see this page on meta.] 65998a13c8fd409fb9002b6d933bdd39c6ec47a8 12 1429 1530 1453 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {{PD Help Page}} You can obtain a copy of the help system in this wiki for local use in your own wiki following these steps: * Step one: In a separate browser window, navigate to [[Special:Export]]. * Step two: You will see an open textbox asking you for a list of pages to export. * Step three: Copy the list below and paste it into that textbox. This is a generally useful set, making the resulting help structure in your wiki look almost identical to the one here. [[Help:Contents]] [[Help:Navigation]] [[Help:Searching]] [[Help:Tracking_changes]] [[Help:Editing_pages]] [[Help:Editing]] [[Help:Starting_a_new_page]] [[Help:Formatting]] [[Help:Links]] [[Help:Categories]] [[Help:Images]] [[Help:Templates]] [[Help:Tables]] [[Help:Variables]] [[Help:Managing_files]] [[Help:Preferences]] [[Help:Skins]] [[Help:Namespaces]] [[Help:Special pages]] [[Help:Range_blocks]] [[Help:Managing_user_rights]] [[Help:Copying]] [[Category:Help]] [[Category:Category]] [[Template:PD_Help_Page]] [[Template:Admin_tip]] [[Template:Prettytable]] [[Template:Hl2]] [[Template:Hl3]] [[Template:Thankyou]] [[:Image:Example.jpg]] [[:Image:Tools.png]] [[:Image:M-en-sidebar.png]] [[:Image:M-en-pagetabs.png]] [[:Image:M-en-userlinks.png]] [[:Image:M-en-recentchanges.png]] Sysops only: [[Template:Meta]] [[Template:Click]] * Step four: Make sure the box "Include only the current revision, not the full history" is CHECKED. * Step five: Click the 'Export' button. * Step six: Save the file to your desktop or other convenient location you'll remember. * Step seven: Navigate to Special:Import on YOUR wiki. (You must be logged in as Admin/Sysop level to do this.) * Step eight: Browse for the file you saved, and click 'Upload'. * Step nine: Download each of the Image files mentioned above (the export/import only copies the image descriptions). * Step ten: Upload each of the Images to the new wiki. * Step eleven: If they haven't come across, manually edit <nowiki>[[Category:Help]]</nowiki> and <nowiki>[[Category:Category]]</nowiki>. Although not complete, at least this gets you started on a decent help system. 3e6e01455883843be45830cbc6365b00b17b8b1d 1575 1530 2007-08-24T08:54:33Z NJAEWikiSysop 1 wikitext text/x-wiki {{PD Help Page}} You can obtain a copy of the help system in this wiki for local use in your own wiki following these steps: * Step one: In a separate browser window, navigate to [[Special:Export]]. * Step two: You will see an open textbox asking you for a list of pages to export. * Step three: Copy the list below and paste it into that textbox. This is a generally useful set, making the resulting help structure in your wiki look almost identical to the one here. Note that the help files on this wiki have been modified from the ones on [http://www.mediawiki.org/wiki/Help:Contents MediaWiki], to remove some interwiki links and the like. [[Help:Contents]] [[Help:Navigation]] [[Help:Searching]] [[Help:Tracking_changes]] [[Help:Editing_pages]] [[Help:Editing]] [[Help:Starting_a_new_page]] [[Help:Formatting]] [[Help:Links]] [[Help:Categories]] [[Help:Images]] [[Help:Templates]] [[Help:Tables]] [[Help:Variables]] [[Help:Managing_files]] [[Help:Preferences]] [[Help:Skins]] [[Help:Namespaces]] [[Help:Special pages]] [[Help:Range_blocks]] [[Help:Managing_user_rights]] [[Help:Copying]] [[Category:Help]] [[Category:Category]] [[Template:PD_Help_Page]] [[Template:Admin_tip]] [[Template:Prettytable]] [[Template:Hl2]] [[Template:Hl3]] [[Template:Thankyou]] [[:Image:Example.jpg]] [[:Image:Tools.png]] [[:Image:M-en-sidebar.png]] [[:Image:M-en-pagetabs.png]] [[:Image:M-en-userlinks.png]] [[:Image:M-en-recentchanges.png]] These files are stubs for pages referred to in the help documentation: [[5 January]] [[05 January]] [[January 05]] [[January 5]] [[01-05]] [[2001]] [[2001-01-05]] [[Dummy date page]] Sysops only: [[Template:Meta]] [[Template:Click]] * Step four: Make sure the box "Include only the current revision, not the full history" is '''checked'''. * Step five: Click the 'Export' button. * Step six: Save the file to your desktop or other convenient location you'll remember. * Step seven: Download each of the Image files mentioned above (the export/import only copies the image descriptions). * Step eight: Navigate to Special:Import on '''your''' wiki. (You must be logged in as Admin/Sysop level to do this.) * Step nine: Browse for the exported file you saved, and click 'Upload'. * Step ten: Upload each of the Images to the new wiki. * Step eleven: If they haven't come across, manually edit [[:Category:Help]] and [[:Category:Category]]. Although not complete, at least this gets you started on a decent help system. 7cc206be2c86ae107f6c332d0e4fbc6960b42b07 1576 1575 2007-08-24T08:55:11Z NJAEWikiSysop 1 wikitext text/x-wiki {{PD Help Page}} You can obtain a copy of the help system in this wiki for local use in your own wiki following these steps: * Step one: In a separate browser window, navigate to [[Special:Export]]. * Step two: You will see an open textbox asking you for a list of pages to export. * Step three: Copy the list below and paste it into that textbox. This is a generally useful set, making the resulting help structure in your wiki look almost identical to the one here. Note that the help files on this wiki have been modified from the ones on [http://www.mediawiki.org/wiki/Help:Contents MediaWiki], to remove some interwiki links and the like. [[Help:Contents]] [[Help:Navigation]] [[Help:Searching]] [[Help:Tracking_changes]] [[Help:Editing_pages]] [[Help:Editing]] [[Help:Starting_a_new_page]] [[Help:Formatting]] [[Help:Links]] [[Help:Categories]] [[Help:Images]] [[Help:Templates]] [[Help:Tables]] [[Help:Variables]] [[Help:Managing_files]] [[Help:Preferences]] [[Help:Skins]] [[Help:Namespaces]] [[Help:Special pages]] [[Help:Range_blocks]] [[Help:Managing_user_rights]] [[Help:Copying]] [[Category:Help]] [[Category:Category]] [[Template:PD_Help_Page]] [[Template:Admin_tip]] [[Template:Prettytable]] [[Template:Hl2]] [[Template:Hl3]] [[Template:Thankyou]] [[:Image:Example.jpg]] [[:Image:Tools.png]] [[:Image:M-en-sidebar.png]] [[:Image:M-en-pagetabs.png]] [[:Image:M-en-userlinks.png]] [[:Image:M-en-recentchanges.png]] These files are stubs for pages referred to in the help documentation: [[5 January]] [[05 January]] [[January 05]] [[January 5]] [[01-05]] [[2001]] [[2001-01-05]] [[Dummy date page]] Sysops only: [[Template:Meta]] [[Template:Click]] * Step four: Make sure the box "Include only the current revision, not the full history" is '''checked'''. * Step five: Click the 'Export' button. * Step six: Save the file to your desktop or other convenient location you'll remember. * Step seven: Download each of the Image files mentioned above (the export/import only copies the image descriptions). * Step eight: Navigate to Special:Import on '''your''' wiki. (You must be logged in as Admin/Sysop level to do this.) * Step nine: Browse for the exported file you saved, and click 'Upload'. * Step ten: Upload each of the Images to the new wiki. * Step eleven: If they haven't come across, manually edit [[:Category:Help]] and [[:Category:Category]] on '''your''' wiki. Although not complete, at least this gets you started on a decent help system. fd1b990187da218f42f08353590465e8c6684468 10 1430 1532 1455 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki <div style="color:#000000; border:solid 1px #A8A8A8; padding:0.5em 1em 0.5em 0.7em; margin:0.5em 0.5em 0.5em 0.5em; background-color:#FFFFFF;font-size:95%; vertical-align:middle;"> {| | These pages are derived from the [http://www.mediawiki.org/wiki/Help:Contents public domain help pages] at [http://www.mediawiki.org/wiki/MediaWiki MediaWiki]. See [[Help:Contents]] for an overview of all Help pages. |} 01fceb84165358212c4e090fed0ad9f20db091d6 10 1431 1534 1457 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki <onlyinclude><div style="font-color:#535068; border:solid 1px #A8A8A8; padding:0.5em 1em 0.5em 0.7em; margin:0.5em 0em; background-color:#FFFFFF;font-size:85%;"> {| |[[Image:Tools.png|20px]] |'''Tip for wiki admins:''' {{{tip}}} |} </div></onlyinclude> [[Category:Info templates|{{PAGENAME}}]] 0221001a20efe5e58a5aaf59da0c13a5b59992ff 10 1432 1536 1459 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki border="2" cellspacing="4" cellpadding="3" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse;empty-cells:show;{{{style|}}}" <noinclude>[[Category:Utility templates|{{PAGENAME}}]]</noinclude> 66042a0bfed4d0e2d45f4bb132266235062a9aa7 10 1433 1538 1461 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki bgcolor="#A7C1F2" <noinclude>[[Category:Utility templates|{{PAGENAME}}]]</noinclude> 3afc2b03bfd3f4c68c0fbb45b7ef52d642b0f8df 10 1434 1540 1463 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki bgcolor="#8DA7D6" <noinclude>[[Category:Utility templates|{{PAGENAME}}]]</noinclude> 37534b6a4b895ec169104a08937b4d37474f1c63 10 1435 1542 1465 2007-08-24T08:40:52Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki <div class="noprint" style="float:right; border:1px solid blue;width:200px;background-color:#fff;padding:3px;"> [[Image:Example.jpg|left|80px|Example sunflower image]] '''A little thank you...''' <br /><small>for {{{1}}}. <br />hugs, {{{2}}}</small> </div> <noinclude>[[Category:Template examples|{{PAGENAME}}]]</noinclude> 42e49911a572c3295ae1828bb8dd932b724da127 6 1436 1544 1467 2007-08-24T08:40:53Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki ---- Note: This image is used as the example image when you use the edit toolbar (English default installation of MediaWiki). ---- * '''Description:''' Sunflowers in Fargo, North Dakota, U.S. * '''Source:''' [http://www.ars.usda.gov/is/graphics/photos/k5751-1.htm www.ars.usda.gov] (cropped and reworked a bit) * '''Author:''' Bruce Fritz * '''Licensing:''' ''This image is in the '''public domain''' because it contains materials that originally came from the Agricultural Research Service, the research agency of the United States Department of Agriculture.'' 3bb0a024af1a31b13b67e93e685dabf8801ac14e 6 1437 1546 1545 2007-08-24T08:40:53Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki Derived from http://www.mediawiki.org/wiki/Image:Tools.svg 11a8b6c88f2197f90475aee334470507537f5c39 6 1438 1548 1471 2007-08-24T08:40:53Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki From http://www.mediawiki.org/wiki/Image:M-en-sidebar.png . Released under version 2 of the [http://www.gnu.org/copyleft/gpl.html GNU General Public License]. f833d106ffae6bc69bdafb18fc38df5b710fe289 6 1439 1550 1473 2007-08-24T08:40:53Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki From http://www.mediawiki.org/wiki/Image:M-en-pagetabs.png . Released under version 2 of the [http://www.gnu.org/copyleft/gpl.html GNU General Public License]. cce112f6ad6d1e761078b8be7decf646ab411fd2 6 1440 1552 1475 2007-08-24T08:40:53Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki From http://www.mediawiki.org/wiki/Image:M-en-userlinks.png bb7614d78c623b8b671c6f3ed4c33eda2847683e 6 1441 1554 1477 2007-08-24T08:40:53Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki From http://www.mediawiki.org/wiki/Image:M-en-recentchanges.png 062b0298ecff4e69cf8c61471fd2abfc76d251b2 10 1442 1556 1479 2007-08-24T08:40:53Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki {| border="0" cellspacing="1" style="background-color:transparent" |- valign="middle" | {{Click|image=Wikimedia-logo.svg | link=meta:{{{1}}} | width=18px | height=18px}} | align="left" | [[meta:{{{1}}}|{{{2|MetaWiki: {{{1}}}}}}]] {{{3|}}} |}<noinclude>[[Category:Info templates|{{PAGENAME}}]]</noinclude> b211224db12655c9ccc050607003ca68e1d541a5 10 1443 1558 1557 2007-08-24T08:40:53Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki <includeonly><div style="position: relative; width: {{{width}}}; height: {{{height}}}; overflow: hidden;"><div style="position: absolute; top: 0px; left: 0px; font-size: 100px; overflow: hidden; line-height: 100px; z-index: 3;">[[{{{link}}}|&nbsp;&nbsp;&nbsp;]]</div> <div style="position: absolute; top: 0px; left: 0px; z-index: 2;">[[Image:{{{image}}}|{{{width}}}|{{{link}}}]]</div> </div></includeonly><noinclude> '''Usage:''' <nowiki>{{Click | image=myimage.png | link=PageName | width=10px | height=10px}}</nowiki> This template attempts to superimpose an invisible link on an image, such that clicking on the image does ''not'' take you to the image description page (normal behaviour). It's useful for small icons/logos which are representative of an article, and which a user might click on expecting to be taken to that article. Here we are using it on the [[Main Page]] for example. It's use should minimised because of the following problems. ==== Problems ==== It doesn't work in text-only browsers, and in screen readers for the disabled, and possibly other situations. The technique of using CSS to change page content also completely breaks an article's web accessibility by contravening a WAI priority-one checkpoint.[http://www.w3.org/TR/WAI-WEBCONTENT/#tech-order-style-sheets] ''Extension:ImageMap'' can achieve the same effect and much more. ==== See also ==== * [http://en.wikipedia.org/wiki/Template:Click Template:Click on wikipedia] where we got this from. * [[Mediazilla:539|<span title="ASSIGNED: Allow images that link somewhere other than the image page">Bug 539</span>]] - Feature request to allow linked images without nasty CSS. </noinclude> e4fed36a3fbd9b2f24271485a4836c9a53b1ca82 0 1446 1560 1559 2007-08-24T08:40:54Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT [[Dummy date page]] 67d6473db21d8588aa0945096e6c3df19957940c 0 1447 1562 1561 2007-08-24T08:40:54Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT [[Dummy date page]] 67d6473db21d8588aa0945096e6c3df19957940c 0 1448 1564 1563 2007-08-24T08:40:54Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT [[Dummy date page]] 67d6473db21d8588aa0945096e6c3df19957940c 0 1449 1566 1565 2007-08-24T08:40:54Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT [[Dummy date page]] 67d6473db21d8588aa0945096e6c3df19957940c 0 1450 1568 1567 2007-08-24T08:40:54Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT [[Dummy date page]] 67d6473db21d8588aa0945096e6c3df19957940c 0 1451 1570 1569 2007-08-24T08:40:54Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT [[Dummy date page]] 67d6473db21d8588aa0945096e6c3df19957940c 0 1452 1572 1571 2007-08-24T08:40:55Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki This is a dummy page used as a stub for the dates link from [[Help:Preferences]]. d681b29046cbd942fb21b0203c47d527e7b06b3d 0 1453 1574 1573 2007-08-24T08:40:55Z NJAEWikiSysop 1 1 revision(s) wikitext text/x-wiki #REDIRECT [[Dummy date page]] 67d6473db21d8588aa0945096e6c3df19957940c 2 1454 1577 2007-08-27T21:40:30Z Neil 2 wikitext text/x-wiki I'm Neil Smith, one of the people who uses this website. e5d7d232c376e03ad921aa8536fe4cad68bd87f4 4 1455 1578 2007-08-27T23:07:01Z Neil 2 Redirecting to [[Help:Searching]] wikitext text/x-wiki #REDIRECT [[Help:Searching]] 71b201df4cbd9886f2647b0bdfdc5393b239a717 0 1 1579 1485 2007-08-28T11:04:00Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]], which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. == Jenni == Content will follow shortly. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. fba1a90eecb64da46c274e3a1a196ce6f2eadd89 1580 1579 2007-08-28T11:05:42Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. == Jenni == Content will follow shortly. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 46d7cfd78a9a0ebe89a7576fa428607e95970518 14 1456 1581 2007-08-28T11:11:20Z Neil 2 wikitext text/x-wiki Blue Planet is a role-playing game of mostly hard science fiction. The publishers are [http://www.biohazardgames.com Biohazard Games]. The game emphasises issues around ecological awareness and biotechnology. This page contains a few original articles, see below. === External sites === There are few Blue Planet websites out there, but their quality more than makes up for a lack of quantity. [http://www.biohazardgames.com Biohazard Games] are the original creators of Blue Planet (though Fantasy Flight Games did the publication for BP v2), and their site contains all sorts of useful and interesting snippets. I find it easier to find things by starting from [http://www.biohazardgames.com/sitemap.html their site map]. The [http://www.visi.com/~seahawk/BP/ Dark Water] site has links to a great number of Blue Planet websites. The sites [http://www.visi.com/~seahawk/BP/misc/index.php 'miscellaneous' page] also got archived copies of two great resources, the OceanView periodical and Mike Z's Blue Planet notebook. OceanView really brings the world to life, and gives that sense that there's a lot more going on than just what your little group is doing. Mike Z's notebook contains all sorts of useful snippets of information and quick-reference tables. SeaHawk also [http://www.geocities.com/seahawk2199/BluePlanet/bpsupport.html has some larger (PDF) files on Geocities], including the first chapter of the Player's Guide, an excellent introduction to the game and the world. [[Category:RPGs]] 8ae7152f94eed26f80f1b5ba87bfb63143039282 1606 1581 2007-10-15T08:50:53Z Neil 2 /* External sites */ Updated Dark Water links wikitext text/x-wiki Blue Planet is a role-playing game of mostly hard science fiction. The publishers are [http://www.biohazardgames.com Biohazard Games]. The game emphasises issues around ecological awareness and biotechnology. This page contains a few original articles, see below. === External sites === There are few Blue Planet websites out there, but their quality more than makes up for a lack of quantity. [http://www.biohazardgames.com Biohazard Games] are the original creators of Blue Planet (though Fantasy Flight Games did the publication for BP v2), and their site contains all sorts of useful and interesting snippets. I find it easier to find things by starting from [http://www.biohazardgames.com/sitemap.html their site map]. The [http://seahawk.110mb.com/ Dark Water] site has links to a great number of Blue Planet websites. The site's [http://seahawk.110mb.com/misc/index.php 'miscellaneous' page] also got archived copies of two great resources, the OceanView periodical and Mike Z's Blue Planet notebook. OceanView really brings the world to life, and gives that sense that there's a lot more going on than just what your little group is doing. Mike Z's notebook contains all sorts of useful snippets of information and quick-reference tables. SeaHawk also [http://www.geocities.com/seahawk2199/BluePlanet/bpsupport.html has some larger (PDF) files on Geocities], including the first chapter of the Player's Guide, an excellent introduction to the game and the world. [[Category:RPGs]] dbdfc23d0acf4a8de4f8d3a2264827735d86bc91 1617 1606 2007-11-26T14:30:00Z Neil 2 /* External sites */ wikitext text/x-wiki Blue Planet is a role-playing game of mostly hard science fiction. The publishers are [http://www.biohazardgames.com Biohazard Games]. The game emphasises issues around ecological awareness and biotechnology. This page contains a few original articles, see below. === External sites === There are few Blue Planet websites out there, but their quality more than makes up for a lack of quantity. [http://www.biohazardgames.com Biohazard Games] are the original creators of Blue Planet (though Fantasy Flight Games did the publication for BP v2), and their site contains all sorts of useful and interesting snippets. I find it easier to find things by starting from [http://www.biohazardgames.com/sitemap.html their site map]. The [http://seahawk.110mb.com/ Dark Water] site has links to a great number of Blue Planet websites. The site's [http://seahawk.110mb.com/misc/index.php 'miscellaneous' page] has archived copies of two great resources, the OceanView periodical and Mike Z's Blue Planet notebook. OceanView really brings the world to life, and gives that sense that there's a lot more going on than just what your little group is doing. Mike Z's notebook contains all sorts of useful snippets of information and quick-reference tables. SeaHawk also [http://www.geocities.com/seahawk2199/BluePlanet/bpsupport.html has some larger (PDF) files on Geocities], including the first chapter of the Player's Guide, an excellent introduction to the game and the world. [[Category:RPGs]] e7b9ae78b18e2a613ee651f9e84307c8783d7e08 0 1457 1582 2007-08-28T11:18:17Z Neil 2 wikitext text/x-wiki In the [[:Category:Blue Planet|Blue Planet]] game, the ''Multiglands'' biological modification allows the user to secrete a number of different hormones and drugs into his system. However, the details of these effects are not described. Here is a brief list of some drugs that could be secreted, and their effects (as inspired by the [http://www.phlebas.com/ Culture novels] of Iain M Banks). ;Snap :A mild, general-purpose stimulant, similar to a large dose of caffine. :''Reduce impairment modifiers from fatigue by 2. +1 Reflexes.'' ;Prang (pre-hangover) :Hangover cure. It alters the body's metabolism of ethanol and methanol, causing the headache and nausea of a hangover to occur immediately, while slightly delaying the elimination of alcohol. Normally glanded just before going to sleep after drinking. When followed by snap the next morning, it can really annoy your friends. ;Flush :Reduces inhibitions and promotes the phsyiological responses of interest, particularly sexual interest (reddened cheeks and lips, dilated pupils, etc.). :''+1 Presence (+2 with opposite sex), -2 Will (for self-control), +2 to all Communication skills.'' ;Glide :A narcotic and relaxant that promotes a feeling of well-being and contentment. :''-1 Intellect, -2 Presence, -1 Agility, acts as Pain Inhibitor (BP:PG p. 135).'' ;Blanch :A comprehensive vaso-constrictor. Its use makes the skin become pale and the extremeties cold. :''+1 Toughness when resisting the effects of hypothermia, acts as Anti-Shock (BP:PG p. 134).'' ;Slice :Reduces the effects of emotions and enhances concentration by divorcing the user from extraneous stimuli. :''+1 Intellect, -2 Presence, +2 Will (for concentration), +2 Awareness for the task at hand, -2 Awareness for everything else, +1 to all Tech skills.'' ;Stalk :Induces a heightened level of alertness. :''+2 Awareness, +2 Reflexes, -1 Intellect (due to distractions).'' ;Cool :Relaxant, inducing a feeling of calm contemplation. :''+2 to tasks involving contemplation.'' ;Flow :Stimulant, increasing levels of motor contol. :''+2 Agility, +2 Dexterity, probable fatigue after use.'' ;Fold :Increases the user's ability to multi-task and empathise with others. :''+1 Awareness, +2 Intuition, -1 Intellect (due to distractions), +2 to all inter-personal skills, reduce penalties due to a busy environment by 2.'' ;Sparkle :A variable-power visual hallucinogen. In mild doses, it causes objects to appear to be surrounded by a sparkling 'aura.' In higher doses, the hallucinations dominate the senses and induce synasthesia. [[Category:Blue Planet]] 472110fd6d6fb25dafd5984784596ee9b2c6b940 1583 1582 2007-08-28T11:19:00Z Neil 2 wikitext text/x-wiki In the [[:Category:Blue Planet|Blue Planet]] game, the ''Multiglands'' biological modification allows the user to secrete a number of different hormones and drugs into his system. However, the details of these effects are not described. Here is a brief list of some drugs that could be secreted, and their effects (as inspired by the [http://www.phlebas.com/ Culture novels] of Iain M Banks). ;Snap :A mild, general-purpose stimulant, similar to a large dose of caffine. :''Reduce impairment modifiers from fatigue by 2. +1 Reflexes.'' ;Prang (pre-hangover) :Hangover cure. It alters the body's metabolism of ethanol and methanol, causing the headache and nausea of a hangover to occur immediately, while slightly delaying the elimination of alcohol. Normally glanded just before going to sleep after drinking. When followed by snap the next morning, it can really annoy your friends. ;Flush :Reduces inhibitions and promotes the phsyiological responses of interest, particularly sexual interest (reddened cheeks and lips, dilated pupils, etc.). :''+1 Presence (+2 with opposite sex), -2 Will (for self-control), +2 to all Communication skills.'' ;Glide :A narcotic and relaxant that promotes a feeling of well-being and contentment. :''-1 Intellect, -2 Presence, -1 Agility, acts as Pain Inhibitor (BP:PG p. 135).'' ;Blanch :A comprehensive vaso-constrictor. Its use makes the skin become pale and the extremeties cold. :''+1 Toughness when resisting the effects of hypothermia, acts as Anti-Shock (BP:PG p. 134).'' ;Slice :Reduces the effects of emotions and enhances concentration by divorcing the user from extraneous stimuli. :''+1 Intellect, -2 Presence, +2 Will (for concentration), +2 Awareness for the task at hand, -2 Awareness for everything else, +1 to all Tech skills.'' ;Stalk :Induces a heightened level of alertness. :''+2 Awareness, +2 Reflexes, -1 Intellect (due to distractions).'' ;Cool :Relaxant, inducing a feeling of calm contemplation. :''+2 to tasks involving contemplation.'' ;Flow :Stimulant, increasing levels of motor contol. :''+2 Agility, +2 Dexterity, probable fatigue after use.'' ;Fold :Increases the user's ability to multi-task and empathise with others. :''+1 Awareness, +2 Intuition, -1 Intellect (due to distractions), +2 to all inter-personal skills, reduce penalties due to a busy environment by 2.'' ;Sparkle :A variable-power visual hallucinogen. In mild doses, it causes objects to appear to be surrounded by a sparkling 'aura.' In higher doses, the hallucinations dominate the senses and induce synasthesia. [[Category:Blue Planet|Multiglands]] 4f78bc496392fd4a8c3ed921a860ce7750389c25 14 1458 1584 2007-08-28T11:20:31Z Neil 2 wikitext text/x-wiki Welcome to my my role-playing pages. Historically, much of my roleplaying is spent in and around Glorantha and similarly, the majority of the material in these pages relates to that. All of my role-playing activity is done at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games Club]. We meet on Tuesday evenings and are always looking for new players and/or GMs. 4fab28b495189672f24d1abbf37441344d354c2a 1585 1584 2007-08-28T11:20:59Z Neil 2 wikitext text/x-wiki Welcome to my my role-playing pages. Historically, much of my roleplaying is spent in and around [[:Category:Glorantha|Glorantha]] and similarly, the majority of the material in these pages relates to that. All of my role-playing activity is done at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games Club]. We meet on Tuesday evenings and are always looking for new players and/or GMs. f393ea5fa1f00d025172a78729b5cc21ce248504 14 1459 1586 2007-08-28T11:25:28Z Neil 2 wikitext text/x-wiki Here are some general thoughts and essays on Glorantha, some rules extensions for RuneQuest, and thoughts on how to run Heroquests. There is more information on Prax, developed for the [[:Category:Yellowtail|Yellowtail campaign]], and the Tertorae clan of the [[:Category:Cinsina|Cinsina tribe]] of Sartar. [[Category:RPGs]] 681c41642a495c305566869bfbed2c40f91b73ab 0 1460 1587 2007-08-28T11:30:01Z Neil 2 wikitext text/x-wiki These are a collection of notes that I've developed while running a couple of Heroquests in my [[:Category:Glorantha|Gloranthan]] role-playing games. I've also included a couple of sample heroquests that I've run. I ran both of these before I got hold of the Heroquesting rules in the Hero Wars/HeroQuest games. Not that that stopped me, as both of these quests were run diceless and effectively mechanicless. The two Heroquests are both 'initiation' type quests, where the questor(s) delved into the deeper mysteries of a god in order to gain extra powers and a greater understanding of their place in the world. The first quest was in a Praxian game, when [[Stars-Not-Shine became a shaman]]. The second quests was for a group of PCs in the Troubled Waters campaign [[Cleansed One heard the Voice of the River|heard the Voice of the River]]. As these were initiation quests, the important factors were to do with the questor's attitude and actions, rather than with raw abilities. From my experience developing and running these quests, I have come up with some advice that I hope others will find useful. My tips for running heroquests are: === Be sure the players trust the GM === Any heroquest is a dangerous undertaking, and these sorts of quests can be the culmination of a major plot for the character. As a GM, the amount of power you have over the character becomes obvious, while many of the in-game limits are removed (the character is, after all, in the Heroplane, where normal rules no longer apply). Before a player will commit his character to a quest, he has to be sure that the GM won't abuse that power, and will make sure the story progresses in an interesting way, regardless of whether the character succeeds or fails in the quest. === Think on several levels === In all quests, there is the overall story that the questor is following, and there is the precise sequence of events that occur. In many cases, the correspondence between these is not obvious, and the questor has to interpret what is happening in terms of the myth. For instance, the events in [[Stars-Not-Shine became a shaman|Stars-Not-Shine's quest]] can be interpreted at three different levels. === Link the events to the questors === What someone finds on the Hero Plane is often a reflection of who they are and what they have done in the mundane world. It makes the quest more personal and immediate if the events in the quest are directly related to the questor's life. === Make internal dialogue external === In all Heroquests, and particularly initiation-style quests, a great focus of the quests lies in how the questor challenges and overcomes their own limitations or connections. Much of this is in terms of a character's internal dialogue. It's more fun for the other players, and adds spice to the questor's player, if this internal dialogue is made explicit. === Ensure the players know the myths === After all this preparation, it helps if the players understand what is going on, and can react appropriately. They won't be able to do this if they don't understand the myths behind the events they're playing through. Either make sure the players have prepared and read all the relevant myths beforehand, or prepare handouts for them. === Don't bother with rules === The two quests I've run were both in games that didn't have rules for Heroquesting. However, in both of these cases, the quests were really about the initiation of the character or characters into some form of deeper mystery about the world. In these cases, the important factor in whether the quest succeeded or not lay in the attitudes of the characters and the actions they attempted. I generally considered that all appropriate actions the characters would take would succeed. The key to the quest lay in what they did and how they acted, and what they learnt about themselves. These are my quests. I hope you enjoy reading them, and that they help to make your next heroquest sessions even more fulfilling and entertaining. * [[Stars-not-Shine became a shaman]] * The [[Cleansed One heard the Voice of the River]] [[Category:RPGs|Heroquests]] 6173eb29cb5061edfee9fc3143c154ffce3fab9a 1588 1587 2007-08-28T11:30:37Z Neil 2 wikitext text/x-wiki These are a collection of notes that I've developed while running a couple of Heroquests in my [[:Category:Glorantha|Gloranthan]] role-playing games. I've also included a couple of sample heroquests that I've run. I ran both of these before I got hold of the Heroquesting rules in the Hero Wars/HeroQuest games. Not that that stopped me, as both of these quests were run diceless and effectively mechanicless. The two Heroquests are both 'initiation' type quests, where the questor(s) delved into the deeper mysteries of a god in order to gain extra powers and a greater understanding of their place in the world. The first quest was in a Praxian game, when [[Stars-Not-Shine became a shaman]]. The second quests was for a group of PCs in the Troubled Waters campaign [[Cleansed One heard the Voice of the River|heard the Voice of the River]]. As these were initiation quests, the important factors were to do with the questor's attitude and actions, rather than with raw abilities. From my experience developing and running these quests, I have come up with some advice that I hope others will find useful. My tips for running heroquests are: === Be sure the players trust the GM === Any heroquest is a dangerous undertaking, and these sorts of quests can be the culmination of a major plot for the character. As a GM, the amount of power you have over the character becomes obvious, while many of the in-game limits are removed (the character is, after all, in the Heroplane, where normal rules no longer apply). Before a player will commit his character to a quest, he has to be sure that the GM won't abuse that power, and will make sure the story progresses in an interesting way, regardless of whether the character succeeds or fails in the quest. === Think on several levels === In all quests, there is the overall story that the questor is following, and there is the precise sequence of events that occur. In many cases, the correspondence between these is not obvious, and the questor has to interpret what is happening in terms of the myth. For instance, the events in [[Stars-Not-Shine became a shaman|Stars-Not-Shine's quest]] can be interpreted at three different levels. === Link the events to the questors === What someone finds on the Hero Plane is often a reflection of who they are and what they have done in the mundane world. It makes the quest more personal and immediate if the events in the quest are directly related to the questor's life. === Make internal dialogue external === In all Heroquests, and particularly initiation-style quests, a great focus of the quests lies in how the questor challenges and overcomes their own limitations or connections. Much of this is in terms of a character's internal dialogue. It's more fun for the other players, and adds spice to the questor's player, if this internal dialogue is made explicit. === Ensure the players know the myths === After all this preparation, it helps if the players understand what is going on, and can react appropriately. They won't be able to do this if they don't understand the myths behind the events they're playing through. Either make sure the players have prepared and read all the relevant myths beforehand, or prepare handouts for them. === Don't bother with rules === The two quests I've run were both in games that didn't have rules for Heroquesting. However, in both of these cases, the quests were really about the initiation of the character or characters into some form of deeper mystery about the world. In these cases, the important factor in whether the quest succeeded or not lay in the attitudes of the characters and the actions they attempted. I generally considered that all appropriate actions the characters would take would succeed. The key to the quest lay in what they did and how they acted, and what they learnt about themselves. These are my quests. I hope you enjoy reading them, and that they help to make your next heroquest sessions even more fulfilling and entertaining. * [[Stars-not-Shine became a shaman]] * The [[Cleansed One heard the Voice of the River]] [[Category:Glorantha|Heroquests]] 217b953825453d7fb47be60b5d561a09f54504bb 0 1461 1589 2007-08-28T11:44:51Z Neil 2 wikitext text/x-wiki This describes the Heroquest I ran for a player whose character wanted to become a shaman. However, readers should be aware of the meta-game context for this quest: the players and myself were all mid to late twenties, and some people had a rather ... direct approach to the less genteel aspects of life. The main player in this quest was seriously considering running Kult as the next game. I don't think he was so sure by the end of this. == The Game == This Heroquest came up as part of the [[:Category:Yellowtail|Yellowtail campaign]], a campaign based in and around the Yellowtail Clan of Praxian Sable Riders. The clan had just returned to Prax from the Wastes, ten years after being forced out for being on the losing side at the Battle of Moonbroth (the Yellowtails had fought against the rest of the Sable nation). On their return, the Yellowtails had been forced into a more anti-Chaos position, with Storm Bull becoming increasingly prominent within the clan. Stars-Not-Shine was the assistant shaman and wanted to become a full shaman. I designed this quest to see if he had the self-discipline and ruthlessness necessary to leave the mundane world behind and live forever with the spirits. == Background == There were two major issues and ongoing plot threads in the campaign at this point that I wanted to include in the story. The first was the movement of the clan towards Storm Bull. Stars-Not-Shine was part of this. While on an expedition to the Big Rubble some time before, Stars-Not-Shine, together with some of his clansmen, had been trapped by a group of Broo. Just when things were getting desperate, Stars-Not-Shine had called on Daka Fal for aid, and was answered when a Storm Khan ancestor of his appeared from the Spirit World and destroyed the Broo. This ancestor spirit, who remained nameless, also helped out Stars-Not-Shine of a number of subsequent occasions. The second involved Milk-of-the-Earth, an upstanding woman in the Yellowtail clan. She had accompanied Stars-Not-Shine and some others from the clan in their search for the Macgu Finn, a powerful fertility spirit that had escaped from the Paps. Stars-Not-Shine and Milk-of-the-Earth successfully recaptured it, but Stars-Not-Shine was greatly affected by the close proximity of the Macgu Finn on the return journey. How greatly affected he was became apparent when they returned to the clan, where it was discovered that Milk-of-the-Earth was pregnant by Stars-Not-Shine. Her husband was not best pleased, and was only just prevented from killing Stars-Not-Shine. This quest took place around the time when Milk-of-the-Earth was due to give birth. == Quest Structure == This quest took place on three levels simultaneously. The deepest level was the standard shaman quest from the RuneQuest rule book: the questor had to find and awaken his fetch, defend himself against the Bad Man, and return to the Inner World. We'd decided that in the Praxian tradition, the shaman's fetch was a spirit ally, not some hidden part of the shaman himself. At the most superficial level, the clan camp was being attacked by a group of Broo, the leader of which was on his own Heroquest. The Broo leader represented the Bad Man for Stars-Not-Shine, and could only be defeated by magical means: he was questing to prevent Stars-Not-Shine finding his fetch. At the intermediate level, the story was one of re-enacting the life of a god. At this level, Stars-Not-Shine had a major choice, which would have ramifications for the remainder of his life. He could choose to follow the path of the Storm Bull, or to follow the path of Daka Fal. Those events are set out below, but note they do not correspond to Heroquest 'stations'. {| ! Daka Fal !! Storm Bull |- | # Be offered various powers and gifts, but refuse them. # Regret these refusals # Be offered Death, and accept it # Notice great problems in the world # Realise these problems are caused by the living and the dead not knowing their places # Separate the living from the dead, placing each in their proper place # Honour and worship the living and the dead, making the world anew | # Fight those you come across # Attack Fear and Death when you find them # Attack the Devil, despite your kin being killed # Be defeated and maimed by the Devil # Accept Eiritha's gift of Life (the fetch), even though it destroys her fertility # Return and defeat the Devil, making the world anew |} (Full details of the [http://www.glorantha.com/library/religions/cult-dakafal.html cult of Daka Fal] and the [http://www.glorantha.com/library/religions/cult-stormbull.html cult of Storm Bull] are on [http://www.glorantha.com/ Issaries Inc's website].) As this was an initiation quest, the major focus of the quest was on Stars-Not-Shine's internal dialogue as he overcame various obstacles in his path. Therefore, as mentioned on the [[How to run a heroquest]] page, I briefed the other players as to the plot, and they played out parts of Stars-Not-Shine's psyche. I decided to present the quest by only describing the events on the mundane level, and allowing the player to interpret these events into whatever myths he wanted. That allowed me to give him the choice of which path to follow at the middle level, without me making it explicit to him. == The Quest == This description is how I designed the quest for Stars-Not-Shine. The [[#What happened|what happened]] section below describes what happened when I ran the quest. === The Journey === The quest starts with the Stars-Not-Shine travelling to the Spirit Plane and seeking out his fetch. As he does so, he comes across some spirits offering him boons. The first is an old man that offers to teach Stars-Not-Shine wisdom if he stays. The second challenge is a nymph that tempts Stars-Not-Shine with pleasures of the flesh. If he succumbs to either of these temptations, he is dumped back in the Inner world, miles from the camp, and needing to purify himself before he can attempt the quest again. === Meeting the Bad Man === After these encounters, Stars-Not-Shine finds himself outside the Yellowtail clan's camp, where a broo war party has gathered. The leader of the broos is obviously Heroquesting as well, and is wielding a greatsword. The broo leader greets Stars-Not-Shine and offers to share his new power, Death, with Stars-Not-Shine. How Stars-Not-Shine reacts at this point will have a major influence on how the remainder of the quest unfolds. It will guide him towards either the Daka Fal or Storm Bull route. This decision isn't final yet, but changing path is not easy. If Stars-Not-Shine chooses to share the new power, he is leaning towards the Daka Fal route. The broo strikes him with the sword and Stars-Not-Shine dies. His body will die in the Inner World, and Stars-Not-Shine's spirit will be wandering without a body. However, this being a Heroquest, Stars-Not-Shine will be able to continue without a body. On the other hand, if he refuses to accept Death, he is veering towards the Storm Bull route, where he fights directly against Chaos. In either case, after the broo leader has spoken to Stars-Not-Shine, he spins away, laughing and dancing, and cuts down one of the Yellowtail braves with a single stroke. The broo will then dance through the camp, slicing open tipis and calling out "Where are you, my precious?" At each tipi, he will strike at the people he finds within, injuring or killing them. The clan's warriors will come out to meet him, but they are severely outclassed, and the best they can do is to reduce the damage the broo can cause. === Following the Storm Bull Path === If he is still alive, Stars-Not-Shine is about the only member of the clan that stand against the broo. While most warriors are incapacitated after a single blow from the broo, Stars-Not-Shine will remain standing, though wounded. This fight should be portrayed as rather desperate, with Stars-Not-Shine's friends and family being maimed or killed around him, with Stars-Not-Shine being attacked and wounded, but no-one being able to really touch the broo, let alone hurt him. Unless he is faced by Stars-Not-Shine, the broo will ignore the questor. During the fight, Stars-Not-Shine should be wounded and knocked to the ground several times. This is another opportunity to play out Stars-Not-Shine's internal dialogue: will he stay to defend the clan, at a great risk to his own safety, or should he leave and ensure his own survival? Stars-Not-Shine should be allowed to struggle back to his feet, and back to the battle, a couple of times, but in a worse condition each time. Eventually, Stars-Not-Shine should be so broken and battered that he cannot continue the fight. At this point, he will find himself outside the tipi where Milk-of-the-Earth is giving birth, but in great pain. === Following the Daka Fal Path === If Stars-Not-Shine accepted the broo's initial offer of Death, he will see the clan's camp in a different way from before. The spirits and the interplay between them will be much more apparent to him. What is obvious to him is a swirling mass of confused spirits swirling around Milk-of-the-Earth's tipi. Leaving his dead body behind, Stars-Not-Shine feels drawn towards this tent. As he does so, he sees the broo attacking the clan, but is powerless to do anything about it. === With Milk-of-the-Earth === Milk-of-the-Earth is in the middle of labour, but there is a problem: the child will not come out. The labour is going so badly that Milk-of-the-Earth is now on the verge of death. What no-one knows is that the child is now dead. If Stars-Not-Shine investigates the spirits that lie within Milk-of-the-Earth, he will discover that the spirit of the dead child is refusing to leave Milk-of-the-Earth's body, which is causing the problems of labour. As soon as Milk-of-the-Earth becomes aware of Stars-Not-Shine, she will cry out to him to take whatever action is necessary to save her and the clan. === The Choice === How Stars-Not-Shine handles this is the key decision in the quest, and defines who becomes Stars-Not-Shine's fetch. If he is dead at this point, it is immediately obvious to Stars-Not-Shine what the problem is: the dead spirit is inside the living mother. If he declined the broo's offer of Death, he can still work this out if he specifically looks at the spirits in and around Milk-of-the-Earth. Once he knows what the problem is, he can talk to the spirit of the stillborn child and attempt to persuade it to leave the land of the living and travel to the land of the dead. After a short time, the spirit agrees to leave the body of the child. At this point, Milk-of-the-Earth's contractions suddenly start to have an effect, and the stillborn baby will be born. However, as the spirit is that of Stars-Not-Shine's child, it maintains a connection to the shaman, and becomes his fetch, embodied in the mummified afterbirth. This is the Daka Fal choice: Stars-Not-Shine has identified the living and the dead and placed each in its proper place. (If Stars-Not-Shine is without a body at this point, because the broo killed it earlier, he can move his spirit into the body of the baby. In this case, Milk-of-the-Earth will give birth to a healthy baby boy that is rather too aware when born and, instead of screaming, will speak to his lover...) If Stars-Not-Shine is alive, he will have been badly beaten up by the broo heroquester. He could follow the Daka Fal route above, but it is more likely that he will follow the Storm Bull route. As soon as Stars-Not-Shine enters Milk-of-the-Earth's tipi, a sensation of intense hunger will join his feelings of pain and fatigue. The scene in Milk-of-the-Earth's tipi is desperate, and Stars-Not-Shine, the only member of the clan who can stand up to the broo, has been defeated again. Milk-of-the-Earth begs him to save herself and the clan by doing whatever is necessary. A quick reminder of the major turning point in the Storm Bull myth, and of his hunger, should prompt Stars-Not-Shine to take the ... appropriate action. After he has eaten his fill on the three kilos of meat, Stars-Not-Shine is newly invigorated and fully healed. He can then go outside and quickly defeat the broo, who will flee. Milk-of-the-Earth is surrounded by enough healers that she won't die from the horrendous wounds inflicted on her by Stars-Not-Shine. However, she will have terrible scars for life on her abdomen, and can never have sex again, let alone another child. Stars-Not-Shine's guardian Storm Khan spirit will reside in a fetish made from the scraps of Milk-of-the-Earth's womb. === Aftermath === Stars-Not-Shine will then return to the mundane world, with his fetch. He will hopefully be seriously questioning whether the price of his shamanhood was too high or not. The rest of the clan may be thinking the same thing. Depending on what fetch he acquired on this quest, Stars-Not-Shine's personality will change slightly; this change will grow as his power and experience develops. If he followed the Storm Bull path, he will become more violent and aggressive. If the followed the Daka Fal path, simply the fact of being a shaman in an infant's body will make him more reliant on others and his own magical abilities. == What happened == Stars-Not-Shine quite quickly dismissed the old man's offer of wisdom and moved on. His encounter with the nymph offering pleasures of the flesh was more fun to play. In this scene we managed to bring out Stars-Not-Shine's internal dialogue. I lovingly described the beautiful, soft, yielding flesh of the nymph, which got Stars-Not-Shine rather hot and bothered. I then went on to describe how she was smiling sweetly and alluringly at him, which got him even more worked up. At this point, one of the other players started muttering comments from Stars-Not-Shine's id into his ear. The scene went something like this... {| !align="right" |GM: |The nymph stares longingly into your eyes. 'Stay here with me,' she whispers, licking her full lips. |- !align="right" |Stars-Not-Shine: |No! I must continue my quest. |- !align="right" |GM: |You're captivated by her curvaceous form. Slowly, her hand draws back her dress to reveal -- |- !align="right" |Other Player: |Phwoarr! Look at that! You're in there! She's begging for it! |- !align="right" |GM: |She smiles coquettishly and beckons you into her chamber. |- !align="right" |Other Player: |Go on! Get in there, my son. You know you want to! |- !align="right" |Stars-Not-Shine: |Oh, it's tempting! Perhaps if I stay for just a while... |- !align="right" |Other Player: |That's the spirit! She's desperate for it -- desperate for you! |- !align="right" |Stars-Not-Shine: |My quest is important! I can't stay here! |} Things carried on like that for a while. Eventually, Stars-Not-Shine decided to leave the nymph behind, but it was only with a great deal of self-discipline and regret. When he met the broos, Stars-Not-Shine decided to decline Death and run to the camp to warn the clan about the impending attack. It took him quite some time to realise that he was the only one who could be effective against the broo leader: he had spent most of his previous career avoiding combat. Eventually, in sheer desperation, he attacked the broo and was defeated. He attacked again and again, and was defeated each time. Things at this point were getting quite desperate. Eventually he was drawn towards Milk-of-the-Earth's tipi. Once there, Stars-Not-Shine didn't know what to do next. A little judicious prompting from me, and pointing out the relevant myths, and Stars-Not-Shine soon knew what he had to do. He also realised the full horror of the sacrifices he was making for his own gain in personal power. When the quest was over, I asked Stars-Not-Shine's player whether it had all been worth it. After all, Stars-Not-Shine was now a shaman, the culmination of several months of play. The player replied, "I'm not sure." That was exactly the result I was hoping for. == Lessons Learnt == Most of the general lessons I learnt from running this quest are listed on the [[How to run a heroquest]] page. The technique of making the internal dialogue external and explicit worked very well, and gave the player several more challenges than would otherwise have been present, particularly in the early stages of the quest. A couple of things didn't go so well. The quest wasn't helped by the main player not reading the various myths before the quest started, as he said he was going to do. I also think that he didn't trust that he could be killed by the broo and still successfully complete the quest. However, the quest did succeed in its main objectives. Stars-Not-Shine became a shaman, and the player realised just what a major change in the character's life this was. Everybody had a good time, much good role-playing had taken place, and there several plot lines had been introduced or developed. All in all, I think it was a success. [[Category:Glorantha]] 54a15240a6b8b8f02de5e62fedd9ab1041cbd108 1591 1589 2007-08-28T11:57:28Z Neil 2 wikitext text/x-wiki This describes the Heroquest I ran for a player whose character wanted to become a shaman. However, readers should be aware of the meta-game context for this quest: the players and myself were all mid to late twenties, and some people had a rather ... direct approach to the less genteel aspects of life. The main player in this quest was seriously considering running Kult as the next game. I don't think he was so sure by the end of this. == The Game == This Heroquest came up as part of the [[:Category:Yellowtail|Yellowtail campaign]], a campaign based in and around the Yellowtail Clan of Praxian Sable Riders. The clan had just returned to Prax from the Wastes, ten years after being forced out for being on the losing side at the Battle of Moonbroth (the Yellowtails had fought against the rest of the Sable nation). On their return, the Yellowtails had been forced into a more anti-Chaos position, with Storm Bull becoming increasingly prominent within the clan. Stars-Not-Shine was the assistant shaman and wanted to become a full shaman. I designed this quest to see if he had the self-discipline and ruthlessness necessary to leave the mundane world behind and live forever with the spirits. == Background == There were two major issues and ongoing plot threads in the campaign at this point that I wanted to include in the story. The first was the movement of the clan towards Storm Bull. Stars-Not-Shine was part of this. While on an expedition to the Big Rubble some time before, Stars-Not-Shine, together with some of his clansmen, had been trapped by a group of Broo. Just when things were getting desperate, Stars-Not-Shine had called on Daka Fal for aid, and was answered when a Storm Khan ancestor of his appeared from the Spirit World and destroyed the Broo. This ancestor spirit, who remained nameless, also helped out Stars-Not-Shine of a number of subsequent occasions. The second involved Milk-of-the-Earth, an upstanding woman in the Yellowtail clan. She had accompanied Stars-Not-Shine and some others from the clan in their search for the Macgu Finn, a powerful fertility spirit that had escaped from the Paps. Stars-Not-Shine and Milk-of-the-Earth successfully recaptured it, but Stars-Not-Shine was greatly affected by the close proximity of the Macgu Finn on the return journey. How greatly affected he was became apparent when they returned to the clan, where it was discovered that Milk-of-the-Earth was pregnant by Stars-Not-Shine. Her husband was not best pleased, and was only just prevented from killing Stars-Not-Shine. This quest took place around the time when Milk-of-the-Earth was due to give birth. == Quest Structure == This quest took place on three levels simultaneously. The deepest level was the standard shaman quest from the RuneQuest rule book: the questor had to find and awaken his fetch, defend himself against the Bad Man, and return to the Inner World. We'd decided that in the Praxian tradition, the shaman's fetch was a spirit ally, not some hidden part of the shaman himself. At the most superficial level, the clan camp was being attacked by a group of Broo, the leader of which was on his own Heroquest. The Broo leader represented the Bad Man for Stars-Not-Shine, and could only be defeated by magical means: he was questing to prevent Stars-Not-Shine finding his fetch. At the intermediate level, the story was one of re-enacting the life of a god. At this level, Stars-Not-Shine had a major choice, which would have ramifications for the remainder of his life. He could choose to follow the path of the Storm Bull, or to follow the path of Daka Fal. Those events are set out below, but note they do not correspond to Heroquest 'stations'. {| ! Daka Fal !! Storm Bull |- | # Be offered various powers and gifts, but refuse them. # Regret these refusals # Be offered Death, and accept it # Notice great problems in the world # Realise these problems are caused by the living and the dead not knowing their places # Separate the living from the dead, placing each in their proper place # Honour and worship the living and the dead, making the world anew | # Fight those you come across # Attack Fear and Death when you find them # Attack the Devil, despite your kin being killed # Be defeated and maimed by the Devil # Accept Eiritha's gift of Life (the fetch), even though it destroys her fertility # Return and defeat the Devil, making the world anew |} (Full details of the [http://www.glorantha.com/library/religions/cult-dakafal.html cult of Daka Fal] and the [http://www.glorantha.com/library/religions/cult-stormbull.html cult of Storm Bull] are on [http://www.glorantha.com/ Issaries Inc's website].) As this was an initiation quest, the major focus of the quest was on Stars-Not-Shine's internal dialogue as he overcame various obstacles in his path. Therefore, as mentioned on the [[How to run a heroquest]] page, I briefed the other players as to the plot, and they played out parts of Stars-Not-Shine's psyche. I decided to present the quest by only describing the events on the mundane level, and allowing the player to interpret these events into whatever myths he wanted. That allowed me to give him the choice of which path to follow at the middle level, without me making it explicit to him. == The Quest == This description is how I designed the quest for Stars-Not-Shine. The [[#What happened|what happened]] section below describes what happened when I ran the quest. === The Journey === The quest starts with the Stars-Not-Shine travelling to the Spirit Plane and seeking out his fetch. As he does so, he comes across some spirits offering him boons. The first is an old man that offers to teach Stars-Not-Shine wisdom if he stays. The second challenge is a nymph that tempts Stars-Not-Shine with pleasures of the flesh. If he succumbs to either of these temptations, he is dumped back in the Inner world, miles from the camp, and needing to purify himself before he can attempt the quest again. === Meeting the Bad Man === After these encounters, Stars-Not-Shine finds himself outside the Yellowtail clan's camp, where a broo war party has gathered. The leader of the broos is obviously Heroquesting as well, and is wielding a greatsword. The broo leader greets Stars-Not-Shine and offers to share his new power, Death, with Stars-Not-Shine. How Stars-Not-Shine reacts at this point will have a major influence on how the remainder of the quest unfolds. It will guide him towards either the Daka Fal or Storm Bull route. This decision isn't final yet, but changing path is not easy. If Stars-Not-Shine chooses to share the new power, he is leaning towards the Daka Fal route. The broo strikes him with the sword and Stars-Not-Shine dies. His body will die in the Inner World, and Stars-Not-Shine's spirit will be wandering without a body. However, this being a Heroquest, Stars-Not-Shine will be able to continue without a body. On the other hand, if he refuses to accept Death, he is veering towards the Storm Bull route, where he fights directly against Chaos. In either case, after the broo leader has spoken to Stars-Not-Shine, he spins away, laughing and dancing, and cuts down one of the Yellowtail braves with a single stroke. The broo will then dance through the camp, slicing open tipis and calling out "Where are you, my precious?" At each tipi, he will strike at the people he finds within, injuring or killing them. The clan's warriors will come out to meet him, but they are severely outclassed, and the best they can do is to reduce the damage the broo can cause. === Following the Storm Bull Path === If he is still alive, Stars-Not-Shine is about the only member of the clan that stand against the broo. While most warriors are incapacitated after a single blow from the broo, Stars-Not-Shine will remain standing, though wounded. This fight should be portrayed as rather desperate, with Stars-Not-Shine's friends and family being maimed or killed around him, with Stars-Not-Shine being attacked and wounded, but no-one being able to really touch the broo, let alone hurt him. Unless he is faced by Stars-Not-Shine, the broo will ignore the questor. During the fight, Stars-Not-Shine should be wounded and knocked to the ground several times. This is another opportunity to play out Stars-Not-Shine's internal dialogue: will he stay to defend the clan, at a great risk to his own safety, or should he leave and ensure his own survival? Stars-Not-Shine should be allowed to struggle back to his feet, and back to the battle, a couple of times, but in a worse condition each time. Eventually, Stars-Not-Shine should be so broken and battered that he cannot continue the fight. At this point, he will find himself outside the tipi where Milk-of-the-Earth is giving birth, but in great pain. === Following the Daka Fal Path === If Stars-Not-Shine accepted the broo's initial offer of Death, he will see the clan's camp in a different way from before. The spirits and the interplay between them will be much more apparent to him. What is obvious to him is a swirling mass of confused spirits swirling around Milk-of-the-Earth's tipi. Leaving his dead body behind, Stars-Not-Shine feels drawn towards this tent. As he does so, he sees the broo attacking the clan, but is powerless to do anything about it. === With Milk-of-the-Earth === Milk-of-the-Earth is in the middle of labour, but there is a problem: the child will not come out. The labour is going so badly that Milk-of-the-Earth is now on the verge of death. What no-one knows is that the child is now dead. If Stars-Not-Shine investigates the spirits that lie within Milk-of-the-Earth, he will discover that the spirit of the dead child is refusing to leave Milk-of-the-Earth's body, which is causing the problems of labour. As soon as Milk-of-the-Earth becomes aware of Stars-Not-Shine, she will cry out to him to take whatever action is necessary to save her and the clan. === The Choice === How Stars-Not-Shine handles this is the key decision in the quest, and defines who becomes Stars-Not-Shine's fetch. If he is dead at this point, it is immediately obvious to Stars-Not-Shine what the problem is: the dead spirit is inside the living mother. If he declined the broo's offer of Death, he can still work this out if he specifically looks at the spirits in and around Milk-of-the-Earth. Once he knows what the problem is, he can talk to the spirit of the stillborn child and attempt to persuade it to leave the land of the living and travel to the land of the dead. After a short time, the spirit agrees to leave the body of the child. At this point, Milk-of-the-Earth's contractions suddenly start to have an effect, and the stillborn baby will be born. However, as the spirit is that of Stars-Not-Shine's child, it maintains a connection to the shaman, and becomes his fetch, embodied in the mummified afterbirth. This is the Daka Fal choice: Stars-Not-Shine has identified the living and the dead and placed each in its proper place. (If Stars-Not-Shine is without a body at this point, because the broo killed it earlier, he can move his spirit into the body of the baby. In this case, Milk-of-the-Earth will give birth to a healthy baby boy that is rather too aware when born and, instead of screaming, will speak to his lover...) If Stars-Not-Shine is alive, he will have been badly beaten up by the broo heroquester. He could follow the Daka Fal route above, but it is more likely that he will follow the Storm Bull route. As soon as Stars-Not-Shine enters Milk-of-the-Earth's tipi, a sensation of intense hunger will join his feelings of pain and fatigue. The scene in Milk-of-the-Earth's tipi is desperate, and Stars-Not-Shine, the only member of the clan who can stand up to the broo, has been defeated again. Milk-of-the-Earth begs him to save herself and the clan by doing whatever is necessary. A quick reminder of the major turning point in the Storm Bull myth, and of his hunger, should prompt Stars-Not-Shine to take the ... appropriate action. After he has eaten his fill on the three kilos of meat, Stars-Not-Shine is newly invigorated and fully healed. He can then go outside and quickly defeat the broo, who will flee. Milk-of-the-Earth is surrounded by enough healers that she won't die from the horrendous wounds inflicted on her by Stars-Not-Shine. However, she will have terrible scars for life on her abdomen, and can never have sex again, let alone another child. Stars-Not-Shine's guardian Storm Khan spirit will reside in a fetish made from the scraps of Milk-of-the-Earth's womb. === Aftermath === Stars-Not-Shine will then return to the mundane world, with his fetch. He will hopefully be seriously questioning whether the price of his shamanhood was too high or not. The rest of the clan may be thinking the same thing. Depending on what fetch he acquired on this quest, Stars-Not-Shine's personality will change slightly; this change will grow as his power and experience develops. If he followed the Storm Bull path, he will become more violent and aggressive. If the followed the Daka Fal path, simply the fact of being a shaman in an infant's body will make him more reliant on others and his own magical abilities. == What happened == Stars-Not-Shine quite quickly dismissed the old man's offer of wisdom and moved on. His encounter with the nymph offering pleasures of the flesh was more fun to play. In this scene we managed to bring out Stars-Not-Shine's internal dialogue. I lovingly described the beautiful, soft, yielding flesh of the nymph, which got Stars-Not-Shine rather hot and bothered. I then went on to describe how she was smiling sweetly and alluringly at him, which got him even more worked up. At this point, one of the other players started muttering comments from Stars-Not-Shine's id into his ear. The scene went something like this... {| !align="right" |GM: |The nymph stares longingly into your eyes. 'Stay here with me,' she whispers, licking her full lips. |- !align="right" |Stars-Not-Shine: |No! I must continue my quest. |- !align="right" |GM: |You're captivated by her curvaceous form. Slowly, her hand draws back her dress to reveal -- |- !align="right" |Other Player: |Phwoarr! Look at that! You're in there! She's begging for it! |- !align="right" |GM: |She smiles coquettishly and beckons you into her chamber. |- !align="right" |Other Player: |Go on! Get in there, my son. You know you want to! |- !align="right" |Stars-Not-Shine: |Oh, it's tempting! Perhaps if I stay for just a while... |- !align="right" |Other Player: |That's the spirit! She's desperate for it -- desperate for you! |- !align="right" |Stars-Not-Shine: |My quest is important! I can't stay here! |} Things carried on like that for a while. Eventually, Stars-Not-Shine decided to leave the nymph behind, but it was only with a great deal of self-discipline and regret. When he met the broos, Stars-Not-Shine decided to decline Death and run to the camp to warn the clan about the impending attack. It took him quite some time to realise that he was the only one who could be effective against the broo leader: he had spent most of his previous career avoiding combat. Eventually, in sheer desperation, he attacked the broo and was defeated. He attacked again and again, and was defeated each time. Things at this point were getting quite desperate. Eventually he was drawn towards Milk-of-the-Earth's tipi. Once there, Stars-Not-Shine didn't know what to do next. A little judicious prompting from me, and pointing out the relevant myths, and Stars-Not-Shine soon knew what he had to do. He also realised the full horror of the sacrifices he was making for his own gain in personal power. When the quest was over, I asked Stars-Not-Shine's player whether it had all been worth it. After all, Stars-Not-Shine was now a shaman, the culmination of several months of play. The player replied, "I'm not sure." That was exactly the result I was hoping for. == Lessons Learnt == Most of the general lessons I learnt from running this quest are listed on the [[How to run a heroquest]] page. The technique of making the internal dialogue external and explicit worked very well, and gave the player several more challenges than would otherwise have been present, particularly in the early stages of the quest. A couple of things didn't go so well. The quest wasn't helped by the main player not reading the various myths before the quest started, as he said he was going to do. I also think that he didn't trust that he could be killed by the broo and still successfully complete the quest. However, the quest did succeed in its main objectives. Stars-Not-Shine became a shaman, and the player realised just what a major change in the character's life this was. Everybody had a good time, much good role-playing had taken place, and there several plot lines had been introduced or developed. All in all, I think it was a success. [[Category:Glorantha]] [[Category:Yellowtail]] d60a827bb90ff8cec7c0b019d52cd70654983edd 0 1462 1590 2007-08-28T11:56:02Z Neil 2 wikitext text/x-wiki This page contains notes of a Heroquest I ran during the Troubled Waters campaign from the River of Cradles supplement for RuneQuest. I was running this game under a late playtest version of the Hero Wars. However, the version I had did not have any rules for Heroquesting in it, so I ran it in the same way as [[Stars-Not-Shine became a shaman|Stars-Not-Shine's quest]]: I did it diceless, nay mechanicless. Again, as this was an initiation-style quest, it was the decisions and attitudes of the characters that was important, not their competence. == Background == The Troubled Waters campaign is excellent for many reasons. The PCs were the 'Chosen Ones' of the Zola Fel cult, chosen to rid the river of all unpleasantness. I was running it for a group of three players, of different role-playing and Gloranthan experience. However, the players were having difficulty getting into the 'We are the Heroes!' mindset required for the campaign to really take off. Instead, they were tending to reject some of the opportunities that were being presented to them. Being a good follower of Joseph Campbell, I decided to give them an explicit 'Call to Adventure' in the hope that this would convince the players and PCs that the PCs really were heroes. What better place to do this than when they arrived at the Shrine of Kinope? When they got there, I decided to ask them to voluntarily become initiates of the Cleansed One subcult of Zola Fel. {| border="1" cellpadding="3" align="right" width="50%" !The Cleansed One Hears the Voice of the River |- |"His eyes opened through the arts of the Three-Bean Circus, the Cleansed One perceived the tragic perversion of his condition and he despaired and would have destroyed himself. He wandered disconsolate and unknowing and coming upon the river Zola Fel, he threw himself into the swift, deep waters that he might drown. He sank to the bottom and tumbled along the river bottom, buffeted by the current. But miraculously, surrounded by darkness and despair, at the brink of death, he discovered within himself a mysterious hope and desire for life. His spirit cried out and was answered by Zola Fel, for suddenly the Cleansed One could see through the dark river depths and breath the pure river waters. "And he felt a hand grasp his right hand. The Cleansed One quailed, for he knew his nature was unclean and he sensed the hand on his was pure and he would withdraw in shame and fear, but the hand held tight and drew him to the banks of the river. And then it seemed to him that on that river bank he slept refreshing sleep of the pure and innocent, yet all that time his spirit was awake and speaking to a Voice. This Voice spoke to him of mysteries and quests, of tasks and trials, of duty and service, of paths that might be travelled to the Heart of the Sea and the secrets of purification that might be found in the deep waters. "And when the Cleansed One awoke, he found he had new faith and purpose. And when he looked upon the palm of his right hand, he saw the marks of the river god's runes, and knew whose hand had taken his darkness, and knew what Voice had spoken to him in sleep. And with the river god's marks he also saw the Rune of Harmony, and knew that this mark was his own, the gift of the insights gathered from the Three-Bean Circus. And by those marks upon his hand he swore forever to be a friend to the river, and to serve it in gratitude for its gift of life and purpose. "So you see how the Cleansed One swore to be a Voice of the River, to praise its gifts of life and purpose, and to preach to all of their duty to serve the river and keep it pure, to respect its gifts that they might not be lost to waste and corruption, and to admonish those who through weakness of wit or poverty of spirit were blind to the great debt owed to Zola Fel by those who shared its bounty." (More details of the [http://www.glorantha.com/library/religions/cult-zolafel.html cult of Zola Fel] are on [http://www.glorantha.com Issaries Inc's website].) |} There were three PCs in the campaign at this point. They were: * Ceoldore Strongfish, an Orlanthi huntsman-turned-mercenary from the Scritha River in Pavis County * Andra, a follower of Chalana Arroy from Pavis * Forazi Netminder, a Zola Fel fisherman from around Pavis Before starting the quest, I asked each player to tell me their PC's greatest hope, greatest fear, and the moments of which they were most and least proud. Not all of these are relevant to the quest, and the specifics of the answers are used below. == Quest Structure == The myth of the Cleansed One is in the box to the right. There were two parts of the story of the Cleansed One that I wanted to stress in this Heroquest. First was the idea of redemption through surrender: by giving up control of his fate, the Cleansed One became greater than he could ever have been otherwise. Second was how the Cleansed One solved problems by bringing together disparate people with different abilities. I was hoping that the players and PCs would learn these lessons and become more heroic in the game. This meant that I had to confront the PCs with a problem that they could not solve for themselves. Instead, they had to realise that individually, they had limits. I could then bring the PCs together and allow them to solve each others' problems. I was also hoping that the act of successfully completing a heroquest for the Zola Fel cult would reinforce the message of heroism in the players and PCs. The myth of the Cleansed One starts with the Cleansed One in the throes of despair, so I had to start with each PC in a position that would cause them to become despairing. This was based on the answers to the questions about what each PC feared the most. The idea was for each of these problems to be caused by another of the PCs, but the solution to cause a problem for another PC. In this way, each problem was insoluble by each PC acting alone. However, if they got together, they would realise that their problems could be solved by each other. As in Stars-Not-Shine's quest, the important part of the quest was the choices the PCs made, so resolution of events in the quest was made without mechanics or dice. Also, events were described to the players in purely mundane terms, which allowed them to interpret things as they saw fit. Learning from previous experience, I gave a copy of the myth to the players to refer to during the quest. == The Quest == This description is how I designed the quest. It covers the major 'stations' in the quest and what was supposed to happen at each. The events section below describes what happened when I ran the quest. === Introduction === The quest started with all the PCs gathering in the Pool at the Shrine of Kinope. Brighteye, the priest of the Cleansed One, met them and led them down into the darkening waters... === Growing Despair === In the first part of the quest, each PC was alone. This section was run separately for each PC, with the other players in another room. {| border="1" cellpadding="3" align="right" width="50%" !The Problems |- | * '''Ceoldore's problem''': for some reason, all the village's crops were vanishing. The hunters had to find food for everyone, but the only food they could find was a slimy snake-like animal. But the more they hunted, the more the crops disappeared * '''Forazi's problem''': something invisible was snatching the fish from the river: they just vanished. In desperation, the riverfolk tried to hide them by turning the river's headwaters into an overgrown swamp. But the more they dug, the more the fish vanished. * '''Andra's problem''': the air was becoming foul, and making people ill. The healers at the temple gave out healing herbs to fend of the effects of the mal aria. But the more herbs they used, the worse the air became. |} Each PC awoke in the home of his parents, exactly as if he had never left home. Fairly soon, he realised that this was a representation of his home in the Hero Plane, and nothing existed outside his own village (or the City of Pavis for Andra). The PC got up, did a normal day's work, went back to bed; no fuss or bother. The same happened the next day. And the next. Then they noticed the problems starting. The problems were specific to each PC, and are listed in the box to the left. Things got worse from day to day, until eventually, in desperation, the community's leader asked for a volunteer to enter the surrounding trackless wastes to find a solution. Of course, the PCs should be those volunteers (this was the 'Call to Adventure' of Joseph Campbell's monomyth, and would hopefully instill the 'heroic' mindset in the PCs and players). === The Three-Bean Circus === Each PC wandered across the trackless waste for a while, until they eventually came across a brightly-coloured tent. This was the home of the Three-Bean Circus. The PC rested there awhile, watching the rehearsals and practice. After a short time, they were noticed and offered some of the Circus's famous salad. The PC, being hungry at this point, ate it. Then the problems started. The PC found themself paralysed, unable to move or speak. The circus performer that gave them the food then started to berate the PC about what a failure they were. This was where the earlier questionairre was useful, as this allowed me to tailor the haranguing to each character specifically. Each PC's fears and failings were used in an attempt to destroy their self-esteem. After a while, the PC was driven the point of despair, recovered the use of their body, and left the Circus. Indeed, each PC "perceived the tragic perversion of his condition and he despaired and would have destroyed himself." The opportunity was found when the PC wandered some more and found himself at the edge of a bottomless chasm that extended forever in all directions. Having failed in their quest to save their community, the PCs jumped to their deaths. As they fell, they fell into a formless void... === The Voice of the River === This was intended to be the easiest part of the quest for me to run. I brought all three players into the room, and then did nothing. The point of this part of the quest was for the PCs to realise that the other PCs were the Voice of the River. By working together, they could realise that each other were the causes and solutions of their problems. The only thing I did was to describe the tugging on their right hands. Oh so gentle at first, it grew in strength as they came closer to a mutual solution. Eventually, when they solved the problem, they realised they were being carried along a rushing stream by the stigmata on their right hands. === The Return === Once the solution was found, they found themselves floating into Kinope's Pool, back where they started. They noticed that their stigmata were a deeper, richer blue, and now permanently damp. They also gained 'Initiate of the Cleansed One' abilities and increases in their 'Breathe Water' and 'See Through Water' abilities. == What happened == The sequence of events described above was followed, pretty much, by each player and PC. The events in the Three Bean Circus were effective, I think because the PC's paralysis added to the dispair the PC was feeling due to the verbal abuse they were getting from the Three Bean Circus performer. Ceoldore's greatest fear was that he was unable to protect and help his family and friends; Andra's fear was that she would fail her patients; and Forazi's fear concerned the feelings he had towards the Yelmalion Templars, with their blond hair, hard muscles, and upright, thrusting spears. Here are some highlights from the quest: * Ceoldore's player was the first to play through the first part of the quest. At the end of the solo section, I described Ceoldore plummeting to his death then said to the player, "We'll leave it there. Send the next one in." The reaction was wonderful: she squealed, "But you can't stop there!", glowered at me, and stomped off into the kitchen to fetch Forazi's player. * Forazi's player didn't really get the idea that, during the Heroquest, Forazi was the Cleansed One. At one point, Forazi lay in the river, spread his arms and called on the Cleansed One for guidance. He took a bit of persuading that the only Cleansed One who could answer his call was himself. * Andra's player told me that Andra's greatest failing was hubris, and she was always portrayed as a headstrong, confident young woman. There was an excellent moment of roleplaying as Andra struggled with herself over jumping into the chasm. She knew it was the right thing to do, but she had great difficulty in admitting defeat and jumping. She eventually did, but only after a great deal of soul-searching. Of the three PCs, Ceoldore got the greatest benefit and Forazi the least. This was based on how well they got 'stuck into' their parts during the solo portion of the quest. == Lessons Learnt == All the players seemed to enjoy the quest, even though it was a mammoth session that went on far too late. They had a real sense of satisfaction when they figured out how their problems were related and what the solutions were. There were comments asking if I'd any experience writing cryptic crosswords, so ingenious did they think the central conundrum. It did achieve the end of increasing the heroic tendencies of the PCs: they were a bit more eager to get stuck in to things after this. In terms of running the quest, I didn't learn much from this one that I hadn't learnt from [[Stars-Not-Shine_became_a_shaman|Stars-Not-Shine's quest]]. These lessons are on the heroquest page. Having a handout of the relevant myth was definitely a good idea, as the players didn't have time to learn the myth, and would have considered it 'homework' in any case. All in all, as successful quest, in all its myriad meanings. == Testemonials == After reading this page, Ceoldore's player wrote: <blockquote>I do NOT squeal!!! Though, on reflection, I do remember something of the sort happening.... <gggg></blockquote> <blockquote>I'd forgotten quite how much fun that was - thank you for running it! It's very interesting to read about your motivation for doing as you did. </blockquote> [[Category:Glorantha]] 911649bee7c8bddd24d6092011bb12a4da93b61a 14 1463 1592 2007-08-28T11:57:59Z Neil 2 wikitext text/x-wiki For a while I was involved with a campaign that was set around the Yellowtail Clan, a Praxian Sable clan that was returning to Prax after leaving for the Wastes after Moonbroth. Because of that, I've developed a reasonable amount of material concerned with Prax and Praxians, some of which is here. [[Category:Glorantha]] f351e2b9dd85cb3efdea04820c9def0b0b837017 0 1464 1593 2007-08-28T12:02:12Z Neil 2 wikitext text/x-wiki ''The Peaceful Cut ceremony is performed by Praxian nomads whenever they slaughter a herd-beast, and most other animals. It shows due deference to the spirit of the animal and guides it on its way back to Mother Erithra. This allows the spirit to be reborn into another animal, and also prevents the spirit haunting the butcher.'' The owner of the animal to be slaughtered (normally a woman) selects the animal from her herd. She approaches the animal and starts to sing a special song to it, to tell the animal what a good beast it is and all about the wonderful place it is going to. This song calms the animal and the woman leads it away to the waiting butcher. She will often give it a message for Mother Erithra just before she hands the animal over to the butcher. Meanwhile, the butcher has selected the spot where the animal will be killed. This should be out of site of the rest of the herds (a gully is ideal) and preferably is somewhere devoid of plants. He has assembled the tools he needs: the hobble-strap, the razor, the hoe, and the axe [1]. He takes the animal from the woman at the point where the animal can no longer see the herd. At this point, the animal is considered dead by the woman and the rest of the clan. The butcher takes the animal to the place of slaughter, singing a calming song. He hobbles the animal so that it cannot move and then chants to the animal to prepare it for its journey back to Mother Erithra [2]. When the time is right, the butcher says, 'The time has come for us to play our parts in the Covenant of Life and Death. We shall remember you in the wealth your body will give us. Return now to Erithra!' and slits the animal's throat with the razor [3]. The blood is allowed to spill to the ground, to provide the sustenance that will allow plants to grow in this spot in the future, to feed more animals. After a few seconds [4], the animal is pushed onto its side and its neck placed over a bowl to collect the rest of the blood. As the blood drains away, the animal's spirit becomes concentrated in its tail. Once the bleeding has stopped, the butcher uses the hoe to dig a pit in the blood-soaked earth and cuts off its tail with a single stroke of the axe. The tail is buried with a prayer to speed the animal's spirit to Mother Erithra. Once this is done, the butchering of the carcass can start. The animal is rolled onto its back and the first cut is along its belly from neck to tail. The skin is layed out to form a mat on which the butchering takes place; this is to prevent anything going to waste. [1] These tools of death are some of a Praxian man's most precious posessions. Good ones are passed down from father to son. [2] Rules note: this singing counts as Ceremony time to increase the butcher's Craft: Butchery skill, at a rate of one minute per increment. [3] Rules note: this is the casting of the Peaceful Cut spell. [4] The longer the animal is allowed to bleed onto the ground, the better luck the tribe will have, but the less food is obtained from the animal. In times of extreme distress, all the animal's blood is allowed to drain away. [[Category:Yellowtail]] da45cca9401aa044d70a03c325bf3b8a6323ebe5 0 1 1594 1580 2007-08-28T13:19:45Z Neil 2 /* Jenni */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 6ba410f06dba44e0fe3bd87a46489c03782219b9 0 1595 1595 2007-08-28T13:33:48Z Neil 2 wikitext text/x-wiki {| border="1" cellpadding="3" align="right" !At A Glance |- | * Chemistry tutoring * A-level specialist * Based in Milton Keynes * £24.00 - £28.00 per hour * Email: [mailto:tutor@njae.me.uk tutor@njae.me.uk] |} I, Jenni Smith, am available to give private tuition in Chemistry at A-level (AS and A2). This tuition can give great support for students, especially for more gifted students who require an additional challenge to achieve their best, or those who could take advantage of extra guidance to achieve higher examination grades. I normally tutor in addition to their normal studies at college or school. My style of tuition is student-led; I concentrate on the areas where the student wants assistance. Most students do well with one or two hours tuition per week, plus some additional work that I set. The earlier tuition starts, the more benefit will be gained. I tutor in the evenings at my house in Milton Keynes. Tutees have come from the area, and from as far afield as Northampton, Wellingborough, and Leighton Buzzard. Please [mailto:tutor@njae.me.uk contact me] if you would like more information, or to arrange tuition. == Subjects Taught == I specialize in A-level tuition, can give tuition for both AS-level and A2-level. I have experience with all exam board syllabi, including Nuffield, Edexcel, AQA, and OCR. == Fees == My fees vary between £24.00 and £28.00 per hour, depending on the level taught. Please [mailto:tutor@njae.me.uk contact me] with your details for more specific fees. == Experience == I am a DfEE qualified teacher with several years' experience of teaching in secondary schools; a large portion of my teaching time was spent teaching Chemistry A-level. I am now teaching special needs children. I have already tutored a number of students at AS and A2 level. I am also an Assistant Examiner for Chemistry A-level, working for the Edexcel board. As with all people who work with children, I have a valid Criminal Records Bureau certificate. References are available on request. 3d3972cc30d634831aea6699fb28906e3da97e3e 1603 1595 2007-09-06T19:34:36Z Neil 2 Updated fees wikitext text/x-wiki {| border="1" cellpadding="3" align="right" !At A Glance |- | * Chemistry tutoring * A-level specialist * Based in Milton Keynes * £27.00 per hour * Email: [mailto:tutor@njae.me.uk tutor@njae.me.uk] |} I, [mailto:tutor@njae.me.uk Jenni Smith], am available to give private tuition in Chemistry at A-level (AS and A2). This tuition can give great support for students, especially for more gifted students who require an additional challenge to achieve their best, or those who could take advantage of extra guidance to achieve higher examination grades. I normally tutor in addition to their normal studies at college or school. My style of tuition is student-led; I concentrate on the areas where the student wants assistance. Most students do well with one or two hours tuition per week, plus some additional work that I set. The earlier tuition starts, the more benefit will be gained. I tutor in the evenings at my house in Milton Keynes. Tutees have come from the area, and from as far afield as Northampton, Wellingborough, and Leighton Buzzard. Please [mailto:tutor@njae.me.uk contact me] if you would like more information, or to arrange tuition. == Subjects Taught == I specialize in A-level tuition, can give tuition for both AS-level and A2-level. I have experience with all exam board syllabi, including Nuffield, Edexcel, AQA, and OCR. == Fees == My fees are £27.00 per hour. == Experience == I am a DfEE qualified teacher with several years' experience of teaching in secondary schools; a large portion of my teaching time was spent teaching Chemistry A-level. I am now teaching special needs children. I have already tutored a number of students at AS and A2 level. I am also an Assistant Examiner for Chemistry A-level, working for the Edexcel board. As with all people who work with children, I have a valid Criminal Records Bureau certificate. References are available on request. 578d336c162991b44511f647c4fbd1b4cb9a0804 1604 1603 2007-09-06T19:45:15Z Neil 2 wikitext text/x-wiki {| align="right" {{Prettytable}} ! {{Hl3}}|'''At A Glance''' |- | * Chemistry tutoring * A-level specialist * Based in Milton Keynes * £27.00 per hour * Email: [mailto:tutor@njae.me.uk tutor@njae.me.uk] |} I, [mailto:tutor@njae.me.uk Jenni Smith], am available to give private tuition in Chemistry at A-level (AS and A2). This tuition can give great support for students, especially for more gifted students who require an additional challenge to achieve their best, or those who could take advantage of extra guidance to achieve higher examination grades. I normally tutor in addition to their normal studies at college or school. My style of tuition is student-led; I concentrate on the areas where the student wants assistance. Most students do well with one or two hours tuition per week, plus some additional work that I set. The earlier tuition starts, the more benefit will be gained. I tutor in the evenings at my house in Milton Keynes. Tutees have come from the area, and from as far afield as Northampton, Wellingborough, and Leighton Buzzard. Please [mailto:tutor@njae.me.uk contact me] if you would like more information, or to arrange tuition. == Subjects and Levels Taught == I tutor Chemistry at A-level. I can give tuition for both AS-level and A2-level. I have experience with all exam board syllabi, including Nuffield, Edexcel, AQA, and OCR. == Fees == My fees are £27.00 per hour. == Experience == I am a DfEE qualified teacher with several years' experience of teaching in secondary schools; a large portion of my teaching time was spent teaching Chemistry A-level. I am now teaching special needs children. I have been tutoring AS and A2 students for over six years and still enjoy it. I am also an Assistant Examiner for Chemistry A-level, working for the Edexcel board. As with all people who work with children, I have a valid Criminal Records Bureau certificate. References are available on request. 9d80ab5d79d2e37228d18ad2a39e118fcca5eaa6 12 1428 1597 1528 2007-09-02T20:34:20Z Neil 2 wikitext text/x-wiki {{PD Help Page}} [http://www.mediawiki.org/wiki/Help:User_rights Please see this page on MediaWiki.] 4722bb886f271decac144077ae643f5cce49d166 1598 1597 2007-09-05T09:37:03Z Neil 2 wikitext text/x-wiki {{PD Help Page}} '''User rights''' are specific access and ability permissions that can be assigned to customizable groups, which can then be assigned to (or removed from) users through the [[Special:Userrights]] interface. For example, the default ''bureaucrat'' group enables the use of the User rights interface. This feature was introduced in MediaWiki 1.5; see [http://meta.wikimedia.org/wiki/Setting_user_rights_in_MediaWiki setting user rights in MediaWiki] on the Meta-wiki for earlier methods. ==Managing groups== A default MediaWiki installation assigns certain rights to default groups (see below). You can change the default rights by declaring the [http://www.mediawiki.org/wiki/Manual:$wgGroupPermissions $wgGroupPermissions array] in [http://www.mediawiki.org/wiki/LocalSettings.php LocalSettings.php] with the syntax "<code>$wgGroupPermissions[' ''group'' '][' ''right'' '] = ''boolean'';</code>". To change permissions for all groups, use "*" as the group. For example, to disable editing for all users, add "<code>$wgGroupPermissions['*']['edit'] = false;</code>". In addition to the default groups, you can arbitrarily create new groups using the same array. For example: $wgGroupPermissions['ninja']['bot'] = true; $wgGroupPermissions['ninja']['block'] = true; $wgGroupPermissions['ninja']['delete'] = true; ===Rights and groups=== The following user rights are available: {| {{prettytable}} |- !{{hl2}} | Right !{{hl2}} | Description |- |{{hl3}} colspan="2"| '''Reading''' |- | read | allows viewing pages not defined in [http://www.mediawiki.org/wiki/Manual:$wgWhitelistRead $wgWhitelistRead]. |- |{{hl3}} colspan="2"| '''Editing''' |- | edit | allows [[help:editing|editing]] unprotected pages. |- | createpage | allows the creation of new pages (requires the ''edit'' right). |- | createtalk | allows the creation of new talk pages (requires the ''edit'' right). |- | move | allows renaming page titles. |- | createaccount | allows the creation of new user accounts. |- | upload | allows the creation of new images and files. |- | reupload | allows overwriting existing images and files. |- | reupload-shared | allows replacing images and files from a shared repository (if one is set up) with local files. |- | upload_by_url | allows uploading by entering the URL of an external image. |- |{{hl3}} colspan="2"| '''Management''' |- | delete | allows the deletion of edits and pages. |- | undelete | allows viewing and restoring deleted edits and pages. |- | deletedhistory | allows viewing deleted revisions, but not restoring. |- | protect | allows locking a page to prevent edits, and editing locked pages. |- | block | allows the blocking of IP addresses, CIDR ranges, and registered users. Block options include preventing editing and registering new accounts, and autoblocking other users on the same IP address. |- | blockemail | allows preventing use of the [[Special:Emailuser]] interface when blocking. |- | userrights | allows the use of the user rights interface, which allows the assignment or removal of groups to any user. |- | rollback | allows one-click reversion of edits. |- | patrol | allows marking edits as legitimate ([http://www.mediawiki.org/wiki/Manual:$wgWhitelistRead $wgUseRCPatrol] must be ''true''). |- | editinterface | allows editing the [[Special:Allmessages|MediaWiki namespace]], which contains [http://www.mediawiki.org/wiki/Manual:Interface interface messages]. |- |{{hl3}} colspan="2"| '''Administration''' |- | siteadmin | allows locking and unlocking the database (which blocks all interactions with the web site except viewing). |- | import | allows user to [http://meta.wikimedia.org/wiki/Help:Import import pages] from another wiki. |- | importupload | allows user to [http://meta.wikimedia.org/wiki/Help:Import import pages] from XML files. |- | trackback | allows removal of [http://www.sixapart.com/pronet/docs/trackback_spec trackbacks] (if [http://www.mediawiki.org/wiki/Manual:$wgUseTrackbacks $wgUseTrackbacks] is ''true''). |- | unwatchedpages | allows access to [[Special:Unwatchedpages]], which lists pages that no user has watchlisted. |- |{{hl3}} colspan="2"| '''Technical''' |- | bot | hides edits from recent changes lists and watchlists by default (can optionally be viewed). |- | purge | allows purging a page without a confirmation step ([http://www.mediawiki.org/wiki/Manual:URL URL parameter] "<code>&action=purge</code>"). |- | minoredit | allows marking an edit as 'minor'. |- | nominornewtalk | blocks new message notification when making minor edits to user talk pages (requires ''minor edit'' right). |- | ipblock-exempt | makes user immune to blocks applied to his IP address. |- | proxyunbannable | makes user immune to the open proxy blocker, which is disabled by default ([http://www.mediawiki.org/wiki/Manual:$wgBlockOpenProxies $wgBlockOpenProxies]). |- | autopatrol | automatically marks all edits by the user as patrolled ([http://www.mediawiki.org/wiki/Manual:$wgUseRCPatrol $wgUseRCPatrol] must be ''true''). |} The following groups are available: {| {{prettytable}} |- !{{hl2}} | Group !{{hl2}} | Description |- | * | all users (including anonymous). |- | user | registered accounts. |- | autoconfirmed | registered accounts at least as old as [http://www.mediawiki.org/wiki/Manual:$wgAutoConfirmAge $wgAutoConfirmAge] and having at least as many edits as [http://www.mediawiki.org/wiki/Manual:$wgAutoConfirmCount $wgAutoConfirmCount]. |- | emailconfirmed | registered accounts with confirmed email addresses. |- | bot | accounts with the ''bot'' right (intended for automated scripts). |- | sysop | users who by default can delete and restore pages, block and unblock users, et cetera. |- | bureaucrat | users who by default can change other users' rights. |} ===Default rights=== The default rights are defined in [http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/includes/DefaultSettings.php?view=markup DefaultSettings.php]. MediaWiki 1.11alpha defines the following: /** * Permission keys given to users in each group. * All users are implicitly in the '*' group including anonymous visitors; * logged-in users are all implicitly in the 'user' group. These will be * combined with the permissions of all groups that a given user is listed * in in the user_groups table. * * Functionality to make pages inaccessible has not been extensively tested * for security. Use at your own risk! * * This replaces wgWhitelistAccount and wgWhitelistEdit */ $wgGroupPermissions = array(); // Implicit group for all visitors $wgGroupPermissions['*' ]['createaccount'] = true; $wgGroupPermissions['*' ]['read'] = true; $wgGroupPermissions['*' ]['edit'] = true; $wgGroupPermissions['*' ]['createpage'] = true; $wgGroupPermissions['*' ]['createtalk'] = true; // Implicit group for all logged-in accounts $wgGroupPermissions['user' ]['move'] = true; $wgGroupPermissions['user' ]['read'] = true; $wgGroupPermissions['user' ]['edit'] = true; $wgGroupPermissions['user' ]['createpage'] = true; $wgGroupPermissions['user' ]['createtalk'] = true; $wgGroupPermissions['user' ]['upload'] = true; $wgGroupPermissions['user' ]['reupload'] = true; $wgGroupPermissions['user' ]['reupload-shared'] = true; $wgGroupPermissions['user' ]['minoredit'] = true; $wgGroupPermissions['user' ]['purge'] = true; // can use ?action=purge without clicking "ok" // Implicit group for accounts that pass $wgAutoConfirmAge $wgGroupPermissions['autoconfirmed']['autoconfirmed'] = true; // Implicit group for accounts with confirmed email addresses // This has little use when email address confirmation is off $wgGroupPermissions['emailconfirmed']['emailconfirmed'] = true; // Users with bot privilege can have their edits hidden // from various log pages by default $wgGroupPermissions['bot' ]['bot'] = true; $wgGroupPermissions['bot' ]['autoconfirmed'] = true; $wgGroupPermissions['bot' ]['nominornewtalk'] = true; $wgGroupPermissions['bot' ]['autopatrol'] = true; // Most extra permission abilities go to this group $wgGroupPermissions['sysop']['block'] = true; $wgGroupPermissions['sysop']['createaccount'] = true; $wgGroupPermissions['sysop']['delete'] = true; $wgGroupPermissions['sysop']['deletedhistory'] = true; // can view deleted history entries, but not see or restore the text $wgGroupPermissions['sysop']['editinterface'] = true; $wgGroupPermissions['sysop']['import'] = true; $wgGroupPermissions['sysop']['importupload'] = true; $wgGroupPermissions['sysop']['move'] = true; $wgGroupPermissions['sysop']['patrol'] = true; $wgGroupPermissions['sysop']['autopatrol'] = true; $wgGroupPermissions['sysop']['protect'] = true; $wgGroupPermissions['sysop']['proxyunbannable'] = true; $wgGroupPermissions['sysop']['rollback'] = true; $wgGroupPermissions['sysop']['trackback'] = true; $wgGroupPermissions['sysop']['upload'] = true; $wgGroupPermissions['sysop']['reupload'] = true; $wgGroupPermissions['sysop']['reupload-shared'] = true; $wgGroupPermissions['sysop']['unwatchedpages'] = true; $wgGroupPermissions['sysop']['autoconfirmed'] = true; $wgGroupPermissions['sysop']['upload_by_url'] = true; $wgGroupPermissions['sysop']['ipblock-exempt'] = true; $wgGroupPermissions['sysop']['blockemail'] = true; // Permission to change users' group assignments $wgGroupPermissions['bureaucrat']['userrights'] = true; // Experimental permissions, not ready for production use //$wgGroupPermissions['sysop']['deleterevision'] = true; //$wgGroupPermissions['bureaucrat']['hiderevision'] = true; /** * The developer group is deprecated, but can be activated if need be * to use the 'lockdb' and 'unlockdb' special pages. Those require * that a lock file be defined and creatable/removable by the web * server. */ # $wgGroupPermissions['developer']['siteadmin'] = true; ==Examples== ===Anonymous users cannot view pages=== This example will disable viewing of all pages not listed in [http://www.mediawiki.org/wiki/Manual:$wgWhitelistRead $wgWhitelistRead], then re-enable for registered users only. $wgGroupPermissions['*']['read'] = false; $wgGroupPermissions['user']['read'] = true; == See also == * [http://www.mediawiki.org/wiki/Manual:$wgNamespaceProtection Manual:$wgNamespaceProtection] * [http://www.mediawiki.org/wiki/Manual:Preventing_access Manual:Preventing access] (examples) * [http://www.mediawiki.org/wiki/Extension:Lockdown Extension:Lockdown] [[Category:Help]] 6f35c6a2bd8fc2c1fcd0e32abdcf043f748a0182 0 1467 1599 2007-09-05T20:43:19Z Neil 2 wikitext text/x-wiki <blockquote>''(NB: this was written before Ancient Echoes was published.)''</blockquote> The canonical BP background has some quite clear-cut moral distinctions, especially between the various Incorporates. GenDiver is an out-and-out baddie, while Hydrospan is portrayed as an ideal, utopian employer. However, I think it's more fun to include some shades of grey into my worlds, so I'd like to make some comments on why all is not happiness and light at Hydrospan. First of all, the idyllic view of Hydrospan, as presented in the BP canon, is the line that the Hydrospan PR people push. But, scratch the surface, and you soon find some darker corners: things may be wonderful, but the psychological stresses placed on its employees, especially its human ones, can be considerable and damaging. Many of the problems in Hydrospan stem from the fact that it is a corporation and a state that is run by dolphins for dolphins. Humans think and behave differently, and this causes problems when they have to fit into a dolphin world. == 30-hour society == Dolphins, being active breathers, don't sleep in the same way that humans do. If they fell completely asleep, they'd suffocate. Instead, dolphins put half of their brain to sleep at a time, meaning that they're awake (at least in part) for the full 30 hours a day. Dolphins really do have a 30-hour society, and it's personal. Each dolphin is awake and active, all hours of the day and night. Unfortunately, humans don't operate this way. It's bad enough that their circadian cycle is disrupted by Poseidon's longer day anyway: but having to keep awake and aware all the time puts a great stress on the human citizens of Hydrospan. But woe betide the human who's asleep when something needs to be done. == Cliques == Humans evolved in groups of up to 100 individuals, and that remains the natural upper limit on sizes of human groups. Dolphins, on the other hand, evolved in groups of only a few dozen or most. This also puts a strain on the human members of Hydrospan: from a human perspective, dolphins are not terribly gregarious, and have a tendency to form small cliques. Once these cliques form, they can put up high social barriers to outsiders, higher than those erected by humans in the equivalent position. This means that humans can very quickly become isolated and given the cold shoulder by their dolphin colleagues. And all these small cliques don't lend themselves to large-scale co-operation and co-ordination, which leads us on to the next problem. == Bullying == In pre-uplift dolphins, small groups of young males would spend much of their time together, but separate from the rest of the pod. These groups would often visit their own, or sometimes other, pods when they wanted something (usually sex). And they'd just take it, using force if necessary. This is a trend which continues in the uplifted dolphins. The dolphin cliques tend to become introverted and isolated from the rest of Hydrospan. Left unchecked, such tendencies would quickly lead to the disintegration of Hydrospan as a corporate entity. The solution is the bully squad. These small groups of young, hot-headed dolphins come down with orders form higher management to make sure that the cliques don't become too introverted and continue to co-operate with the rest of the corporate state. They do this by very assertive, sometimes violent, means. They barge into the clique, demand what information they've been tasked to get, and throw their weight around to ensure that Hydrospan continues to operate in a smooth manner. And if that means that examples must be made, and cliques broken up, so be it. Even if long-term friendships or family ties are shattered in the process. == Up or Out == Hydrospan makes a big deal of its meritocratic structure: everyone starts at the bottom, and is promoted solely on the basis of performance. If you're good, you go up. If you're not, you don't. That would be fine, except that performance assessment applies to everything. This creates pressures on all the employees, and can lead to some rather creative reporting of outcomes and achievements. It also leads to some really quite vicious infighting as rivals vie for the one promotion on offer. It's a trend which could easily lead to corruption and eventual corporate destruction. And it all adds pressure to the already stressed humans. == Long-term relationships and sex == The final difference between dolphins and humans is in sex and long-term relationships. Humans pair-bond for long periods, often life. Sex is normally restricted (by social mores) to couples that have long-term relationships. And there's a physiological support for that: post-coital hormone surges reinforce the emotional attachments to the other in the pair-bond. Dolphins don't to this. To them, sex is far more casual, almost a way of greeting. Dolphins don't really understand the concept of long-term monogamy. To them, promiscuity is natural and a good thing. It means that if a human does form a long-term, monogamous relationship, they're regarded as rather odd, perhaps even unhealthy. Humans are expected to change partners often, despite the emotional costs to the individuals involved. == Ramifications == All of this means that a great many humans become damaged by their tenure in Hydrospan. The effects are less in those people that have grown up in the Incorporate state, but even these humans find that the dolphins' ways can be disquieting, even unpleasant. The 30-hour society leads to the widespread use of stimulants by humans. There are times when they just have to keep going for long periods, perhaps days at a time, without sleep. The only way to achieve this is pharmaceutically, whether by external drugs or implanted multiglands. Bullying is a fact of life. When the bully squad comes to visit (and they will), expect a hard time. And even when they're not around, the constant pressure on getting results makes for a stressful working environment. Finally, the pressure to dissolve long-term relationships can become intolerable. Many a human in Hydrospan has had to make the difficult choice between leaving their spouse, and perhaps children, or seeing their career slam into a dead end. All these stresses lead to some severe psychological damage among Hydrospan's human employees. To avoid squandering their human capital, Hydrospan has a vigorous and comprehensive mental health programme. Most of the time, this take the form of counselling and psychological assessment, but it quite often steps over into formal psychiatric treatment. Most high-ranking human employees have had mind-jobs of various degrees, to allow them to cope with the extraordinary pressures that arise from living in a dolphin world. But these treatments aren't always successful, and sometimes early 'retirement' to a controlled environment is the only option. That the damage Hydrospan can cause to its citizens is not more widely known is a testament to its publicity and marketing machine. Aggressive, though subtle, information campaigns are constantly run by Hydrospan both to present itself in the best possible light, and to turn attention onto other issues. The effectiveness of these campaigns is largely due to the wealth of experience of manipulating human nature from which Hydrospan can draw: experience that largely arose from dealing with its own problems. == Conclusion == So, what is the true face of Hydrospan? Is it the utopian ideal that is presented, or is that simply a gossamer of PR that obscures something altogether more sinister and unpleasant? I think it's something for you to discover through your own games. [[Category:Blue Planet|Darker side of Hydrospan]] 79b07081786c8e5d54f2b4b71423bd74d6c89185 0 1468 1600 2007-09-05T20:50:42Z Neil 2 wikitext text/x-wiki The 30-hour day on Poseidon takes some getting used to, both for Terran characters and players. In an attempt to ease this, here is my take on the rhythm of a typical day. {| cellpadding="3" |- | 0000: || Midnight. Most people asleep. |- | 0700: || Dawn. Wake up. Breakfast. |- | 0900: || Nominal start of "working day". |- | 1200: || Short break. Snack. |- | 1500: || Noon. Light lunch. Start of siesta. |- | 1900: || Mid-afternoon. End of siesta. |- | 2300: || Dusk. Evening. Nominal end of "working day". |- | 2400: || Main meal eaten (family), or start of evening entertainment. |- | 2600: ||Evening entertainment. Generally promenading, people-watching, visiting "tapas" bars, and the like. |- | 2900: || Night. Most people to bed. |- | 3000: || Midnight again. |} Timekeeping over a longer period is covered in my description of the [[Poseidon calendar]]. [[Category:Blue Planet|Day on Poseidon]] 11bd6f25866dfa931f2d48b8ec67fb0caa68f124 0 1469 1601 2007-09-05T20:58:54Z Neil 2 wikitext text/x-wiki {| {{Prettytable}} |- | &nbsp; || {{Hl2}}|''Siderial'' || {{Hl2}}|''Synodic'' |- | {{Hl3}}|'''Day length''' || 30 h, 0 m, 43 s || 30 h, 6 m, 9 s |- | {{Hl3}}|'''Proteus "month"''' || 1166 h, 47 m, 25 s || 43 d, 26 h, 51 m, 50 s |- | {{Hl3}}|'''Nereus "month"''' || 1942 h, 06 m, 19 s || 80 d, 03 h, 11 m, 2 s |- | {{Hl3}}|'''Length of year''' || 9979 h, 27 m, 14 s || 331 d, 15 h, 31 m, 1 s |} (Siderial times are taken relative to the fixed stars; synodic (and solar) times are taken relative to the position of the sun overhead. Syndoic times are given in terms of local (solar) days.) As the Poseidon year is a fraction over 331½ days, leap years occur more often than on Earth. The rule is: * Odd-numbered years are leap years (332 days) * Years that are multiples of 100 are leap years (332 days) * Years that are multiples of 200 are double leap years (333 days) Other, archaic, units of time are still used. The week is still defined to be seven days, and the month is defined as 30 days. As for how Posiedoners spend their 30-hour days, another page describes the rhythm of [[a day on Poseidon]]. [[Category:Blue Planet]] e065000c8ec930ee8014210b35875d865c0a2f73 0 1481 1602 2007-09-05T21:04:30Z Neil 2 wikitext text/x-wiki I use a few house rules when running [[:Category:Blue Planet|Blue Planet]]. They are all minor, and have very little effect on the already lightweight Synergy game system. == Character Generation == I made a few restrictions on the character generation choices available to the players. * Characters are at the Exceptional power level, apart from having Everyday attributtes (i.e. sum to zero) * There are no systemic osmoform aquaforms (squids): gills on a mammal would cause virtually instant and terminal hypothermia. Diving reflex aquaforms, however, are perfectly believable. * Hybrids look far more like normal humans than how they are presented in the rules: genetic modification to increase muscle mass, say, will have no effect on the development of, say, the nose. Silvas tend to look like people with acromegaly, with large jaws and hands and increased muscle mass. Cats are thin, nervous, and constantly jumpy. == Open-ended dice rolls == When making a task attempt, natural rolls of '1' and '10' are open ended. If the lowest die rolled is a '1', add 10 to the Target Number and roll again (but only reroll the dice that show a '1'). If the lowest die rolled is a '10', subtract 10 from the Target number and roll again. Multiple consecutive rolls of '1' or '10' keep moving the Target Number. This mechanism allows attempts at tasks that, through various factors, have negative target numbers. :(Inspired by Mike Z's house rule) == Augmentation == * When two or more skills are applicable to a certain task (e.g. Pilot and Computer when trying to override a sabotaged autopilot), the secondary skill can give a bonus of one-third its value (round down) to the main skill value. :(Inspired by [[:Category:HeroQuest|HeroQuest]]) == Close Combat == First is the single erratum from the published rulebooks: * Unarmed comabat strikes have a damage rating of Strength + 1 Second is the house rule. * When making an offensive action (Strike or Throw), but before dice are rolled, an attacker can nominate to forgo an attempt to cause damage, but instead acquire a combat advantage. In this case, the attacker's net Action Value (attacker's AV - defender's AV) is applied as a bonus to the attacker's next action. Note that the net AV can be negative, giving the attacker a penalty. Also note that the combat advantage resets to zero if it is not used in the attacker's next action. == Equipment == * The electronic weapon scope does not give a bonus to all shots; instead, it alleviates any range penalites by 2. Its range-trebling effect remains unchanged. * The targeting interlink does not interface with programmed reflexes to provide a 'never miss' effect. It still gives a +2/+3 bonus to all shots when used. == Biomods == * The diver aquaform includes the improved blood oxygenation modification. Characters with either modification have the union of the stated benefits: they can hold their breath for up to 10 minutes while performing strenuous activity, and up to an hour if resting. * Multiglands can produce a number of effects. See [[what multiglands can do for you]] for details. [[Category:Blue Planet]] 37c656ef3df5b62ea85c91b6c0ebad161a2cea9f 1623 1602 2007-11-26T21:42:23Z Neil 2 [[Blue Planet house rules]] moved to [[Synergy house rules]] wikitext text/x-wiki I use a few house rules when running [[:Category:Blue Planet|Blue Planet]]. They are all minor, and have very little effect on the already lightweight Synergy game system. == Character Generation == I made a few restrictions on the character generation choices available to the players. * Characters are at the Exceptional power level, apart from having Everyday attributtes (i.e. sum to zero) * There are no systemic osmoform aquaforms (squids): gills on a mammal would cause virtually instant and terminal hypothermia. Diving reflex aquaforms, however, are perfectly believable. * Hybrids look far more like normal humans than how they are presented in the rules: genetic modification to increase muscle mass, say, will have no effect on the development of, say, the nose. Silvas tend to look like people with acromegaly, with large jaws and hands and increased muscle mass. Cats are thin, nervous, and constantly jumpy. == Open-ended dice rolls == When making a task attempt, natural rolls of '1' and '10' are open ended. If the lowest die rolled is a '1', add 10 to the Target Number and roll again (but only reroll the dice that show a '1'). If the lowest die rolled is a '10', subtract 10 from the Target number and roll again. Multiple consecutive rolls of '1' or '10' keep moving the Target Number. This mechanism allows attempts at tasks that, through various factors, have negative target numbers. :(Inspired by Mike Z's house rule) == Augmentation == * When two or more skills are applicable to a certain task (e.g. Pilot and Computer when trying to override a sabotaged autopilot), the secondary skill can give a bonus of one-third its value (round down) to the main skill value. :(Inspired by [[:Category:HeroQuest|HeroQuest]]) == Close Combat == First is the single erratum from the published rulebooks: * Unarmed comabat strikes have a damage rating of Strength + 1 Second is the house rule. * When making an offensive action (Strike or Throw), but before dice are rolled, an attacker can nominate to forgo an attempt to cause damage, but instead acquire a combat advantage. In this case, the attacker's net Action Value (attacker's AV - defender's AV) is applied as a bonus to the attacker's next action. Note that the net AV can be negative, giving the attacker a penalty. Also note that the combat advantage resets to zero if it is not used in the attacker's next action. == Equipment == * The electronic weapon scope does not give a bonus to all shots; instead, it alleviates any range penalites by 2. Its range-trebling effect remains unchanged. * The targeting interlink does not interface with programmed reflexes to provide a 'never miss' effect. It still gives a +2/+3 bonus to all shots when used. == Biomods == * The diver aquaform includes the improved blood oxygenation modification. Characters with either modification have the union of the stated benefits: they can hold their breath for up to 10 minutes while performing strenuous activity, and up to an hour if resting. * Multiglands can produce a number of effects. See [[what multiglands can do for you]] for details. [[Category:Blue Planet]] 37c656ef3df5b62ea85c91b6c0ebad161a2cea9f 1625 1623 2007-11-26T21:43:30Z Neil 2 wikitext text/x-wiki I use a few house rules for the Synergy system, as used in [[:Category:Blue Planet|Blue Planet]]. They are all minor, and have very little effect on the already lightweight Synergy game system. == Character Generation == I made a few restrictions on the character generation choices available to the players. * Characters are at the Exceptional power level, apart from having Everyday attributtes (i.e. sum to zero) * There are no systemic osmoform aquaforms (squids): gills on a mammal would cause virtually instant and terminal hypothermia. Diving reflex aquaforms, however, are perfectly believable. * Hybrids look far more like normal humans than how they are presented in the rules: genetic modification to increase muscle mass, say, will have no effect on the development of, say, the nose. Silvas tend to look like people with acromegaly, with large jaws and hands and increased muscle mass. Cats are thin, nervous, and constantly jumpy. == Open-ended dice rolls == When making a task attempt, natural rolls of '1' and '10' are open ended. If the lowest die rolled is a '1', add 10 to the Target Number and roll again (but only reroll the dice that show a '1'). If the lowest die rolled is a '10', subtract 10 from the Target number and roll again. Multiple consecutive rolls of '1' or '10' keep moving the Target Number. This mechanism allows attempts at tasks that, through various factors, have negative target numbers. :(Inspired by Mike Z's house rule) == Augmentation == * When two or more skills are applicable to a certain task (e.g. Pilot and Computer when trying to override a sabotaged autopilot), the secondary skill can give a bonus of one-third its value (round down) to the main skill value. :(Inspired by [[:Category:HeroQuest|HeroQuest]]) == Close Combat == First is the single erratum from the published rulebooks: * Unarmed comabat strikes have a damage rating of Strength + 1 Second is the house rule. * When making an offensive action (Strike or Throw), but before dice are rolled, an attacker can nominate to forgo an attempt to cause damage, but instead acquire a combat advantage. In this case, the attacker's net Action Value (attacker's AV - defender's AV) is applied as a bonus to the attacker's next action. Note that the net AV can be negative, giving the attacker a penalty. Also note that the combat advantage resets to zero if it is not used in the attacker's next action. == Equipment == * The electronic weapon scope does not give a bonus to all shots; instead, it alleviates any range penalites by 2. Its range-trebling effect remains unchanged. * The targeting interlink does not interface with programmed reflexes to provide a 'never miss' effect. It still gives a +2/+3 bonus to all shots when used. == Biomods == * The diver aquaform includes the improved blood oxygenation modification. Characters with either modification have the union of the stated benefits: they can hold their breath for up to 10 minutes while performing strenuous activity, and up to an hour if resting. * Multiglands can produce a number of effects. See [[what multiglands can do for you]] for details. [[Category:Blue Planet]] 116de5886ed0c921d843e29e3ad5fcd00fd99a5b 0 1471 1605 2007-09-10T19:26:52Z Neil 2 wikitext text/x-wiki The writeups for ''[http://www.mk-rpg.org.uk/Longshore_Crew The Longshore Crew]'' campaign have moved to the [http://www.mk-rpg.org.uk/Main_Page MK RPG web page]. [[Category:Blue Planet]] dc640620d8dff0e16d0869d7c588e0a39ae6edb6 0 1472 1607 2007-11-12T20:47:10Z Neil 2 wikitext text/x-wiki (As told by Three Cows, Brave of the Yellowtail Clan) 'When the Lunar missionaries first came to Prax, the True People were unsure how to treat them. They were outsiders, dirtdiggers and so deserved contempt. But they had many gifts of metal to buy their way into tribes where they could use the magic of their clever tongues on the True People. But this could not hide the fact that their Red Queen did not follow the Covenant of Waha, and that she consorted with Chaos. These facts meant that most True People rejected the Red Men and killed them when they were found. In that way, we took their metal gifts without being corrupted by their clever tongues. But the Great Council of the Sable Nation heard too much of the Red Man's twisted words and agreed to help them, for many gifts of metal, magic, and salt. Only our Khan, Sun-Through-Clouds, spoke against them. He was forced out of the council tipi. 'In this way, when the Red Man's army came to Moonbroth, all the Sable Nation fought alongside them, except for the Yellowtail Clan, true followers of the Way of Waha. The battle went badly for the Red Men did not fight with honour. When the shamans summoned Oakfed, the Wildfire, the Lunars cast a terrible magic and sent Oakfed back against our shamans. Only because he was a stout follower of Waha and Daka Fal was Nine Sticks saved on that day, though he has not been right since. 'After the defeat of the True Peoples, the Sable Nation turned on our clan as traitors to the Nation. But they were the traitors! Bereft of allies, we were forced out of the good lands of Prax back out into the Wastes. Before we left, many men left and took their wives and herds with them, to join other Sable clans, may their names be cursed. 'That happened two-hands-and-one years ago. We have been in the Wastes all that time. Our clan is small. We are too small to prevent other tribes stealing our cows: that is why our herds are small. We are too small to aid Nine Sticks in driving away Malia: that is why our herds are sick, and why hoof-canker took many of them. That is why many babies died from the Shakes. We are too small to drive other clans from good grazing: that is why our cows are thin. Khan Sun-Through-Clouds has decided that we have spent too long in the Wastes, and that we must return to Prax, where we can steal cows, find good grazing, and bring up our children.' You can also read about the [[People of the Yellowtail Clan]]. [[Category:Yellowtail]] 58d53054f4410c25ab2cba4f147350287cf796aa 0 1473 1608 2007-11-12T20:54:30Z Neil 2 wikitext text/x-wiki == Notable Personalities == === Khan Sun-Through-Clouds === (Wise, stoic, conservative) Khan Sun-Through-Clouds is almost the oldest man in the tribe, having lived for more than two-hands-of-hands of winters. He was once a great warrior and a wise Khan, well respected amongst the elders of the Sable Nation. But now he is old and not a great warrior. He once had a whole herd of captured beasts, a glorious thing, but he gave them to the tribe when times were hard and now he only owns two war sables, five impalas, two bison, four herd-men, and an ostrich. He also owns an iron-headed lance, a mighty bow, a stone that smells bad when Chaos is near, and an eagle head-dress. His lance is seldom used now and he can no more string the bow. It should belong to his sons, but they were killed at Moonbroth. His successor will take it when he dies. Sun-Through-Clouds has lead the tribe for many hands of years. He has lead the tribe well, protecting the herds and stealing cows from other tribes. He stood by his principles when the Sable Nation decided to join the Red Man's army, although it was an unpopular decision. He was forced to head out into the wastes with only half a clan; the other half decided to stay and join other Sable tribes. Normally, Sun-Through-Clouds says little in tribal council meetings. He is happy to let Queen Many-Feet make the decisions about the day-to-day running of the tribe, according to the Covenant of Waha. He only interjects when he feels his role as protector is necessary. During the men's meetings, he also speaks little, preferring to allow the younger men to make speeches and put forward alternative courses of action. He allows them to argue until feelings either run too high or the argument is finished. When he does speak, everyone listens as he gives wise words and acts to balance the factions in the tribe, to preserve what little unity there is left. He is expected to die soon. Because of this, and because he has no sons to succeed him, many braves are vying for position to be the next Khan. They are split into two factions: the pro-Lunars, led by Queen Many-Feet (with the brave Hyena-Tail as her spokesman) and the anti-Lunars, led by Little Mouse, the Storm Khan. At the moment, the pro-Lunars are ascendant within the clan as they have persuaded Sun-Through-Clouds to return to Prax. Many braves are allying themselves to one or other of these factions. Sun-Through-Clouds expects the clan to split again when he dies, and he prays that he can survive and keep the clan together until it can regain strength in Prax. === Rani Many-Feet === (Wilful, protective, manipulative) This woman is the herd-mother, comforter, healer and guide of the people. She is skilled in the lore of Sables and can find water and fodder in even trackless desert. However, it is often that the oases are occupied by other, stronger clans. Her sole passion is for the health and safety of the herds and she will endure any hardship and undergo any trial to preserve them. She feels that she is the best able to lead the clan and that its destiny lies back in Prax, no longer an outcast from the Sable Nation. She has persuaded Sun-Through-Clouds to lead the clan back there and is encouraging support within the clan for her plan to join the Lunar sympathisers. === Nine Sticks, Speaker-to-Spirits === (Deranged, Delusional, Impertinent) This shaman is the clan's speaker to spirits and its ancestors. He deliberately sets himself apart from the remainder of the clan, preferring the company of the inhabitants of the Other Side to his own kind. To show his separation, he dresses in the clothes of a woman. He fought at Moonbroth, and only survived the Lunar treachery because he know the secrets of Inora, the White Princess. === Limping Bull, Storm Bull Berserk === (Fanatical, arrogant, violent) He leads the Running Dogs society, open to only those men that have killed Chaos; there are three members at the moment. This society also acts as 'police' in the clan. There are moves to expand the membership of this society, following the massacre of the clan's Storm Bullers. === Bat's Eye === (Eloquent, companiable, scathing) This ageing brave acts as Khan Sun-through-Clouds' eyes and ears throughout the clan. People don't mind this, as he is always friendly and his tipi is always filled with people eager to hear is stories of far-away lands and strange people. He often uses these occasions to feel the mood of the men, and uses his quick tongue to chastise those that have not acted like a true warrior. Due to his way with words, he is also the clan's herald. He is the one that goes out to meet and parley with other clans when we meet. === Hyena-Tail === (Weak-willed, frustrated, eloquent) This brave is the mouthpiece of Queen Many-Feet in the men's council. He is despised for being so weak-willed as to be lead by a woman, but many people agree with what he says. He is not a great warrior, but is a competent hunter and has captured some enemies' beasts. == Distinguishing Clan Features == * ''Animals'': yellow-stained tail (the yellow die is obtained from boiling down Sable gall-bladders). * ''Men'': yellow feathers in headband, zigzag tattoos on cheeks. * ''Women'': twice-braided hair, yellow stripe down the back of tunics. == Spirits worshipped == {| Spirit || Highest ranking member || Other high-lvl || Other initiates || Lay members || Temple size |- |Waha || Sun-Through-Clouds (Khan) || 1 || 15 || 5 || Small |- |Erithra || Many Feet (Rani) || 2 || 24 || 10 || Small Daka Fal || Nine Sticks (Shaman Priest) || 1 || 1 || 3 || Shrine Storm Bull || Limping Bull (Initiate) || 0 || 0 || 2 || None* Foundchild || Fallen Branch (Initiate) || 0 || 4 || 5 || Shrine Chalana || Flowers' Scent (Acolyte) || 0 || 2 || 4 || Shrine Issaries || Bat's Eye (Acolyte) || 0 || 1 || 2 || None Lankoring || Small Hawk (Lay member) || 0 || 0 || 0 || None Humakt Antak-Dor-Tak (?) || 0 || 0 || 1 || None Eurmal Trail's End (Initiate) || 0 || 0 || 13 || None * The shrine to Storm Bull is temproarily inactive, due to a near-zero congregation. Demographics of the Yellowtail Clan Ages Male Female Total 0-1 1 1 2 1-5 3 3 6 5-10 3 3 6 10-15 2 3 5 15-20 2 3 5 20-25 2 3 5 25-30 2 3 5 30-35 2 2 4 35-40 1 2 3 40-45 1 2 3 45-50 1 1 2 50-55 1 1 2 55+ 1 1 2 Totals 22 28 50 Plus 6 female slaves aged 20-30 Current Herd Required Herd Herd size 400 (560) Males 40 (56) Females 360 (504) Males born 61 (120.20) Females born 54.1 (106.60) Male calves dying 42.7 (60.10) Female calves dying 37.9 (53.30) Male adults dying 4 (5.6) Female adults dying 36 (50.4) Dead animals (adult equiv) 80.3 (112.70) Net herd gain -5.4 (57.4) Females giving milk 208.8 (226.8) You can read the [[history of the Yellowtail Clan]]. [[Category:Yellowtail]] 59eee2d3a610f59f46a097d4c381fc8a8e82db36 1609 1608 2007-11-12T22:14:34Z Neil 2 /* Spirits worshipped */ wikitext text/x-wiki == Notable Personalities == === Khan Sun-Through-Clouds === (Wise, stoic, conservative) Khan Sun-Through-Clouds is almost the oldest man in the tribe, having lived for more than two-hands-of-hands of winters. He was once a great warrior and a wise Khan, well respected amongst the elders of the Sable Nation. But now he is old and not a great warrior. He once had a whole herd of captured beasts, a glorious thing, but he gave them to the tribe when times were hard and now he only owns two war sables, five impalas, two bison, four herd-men, and an ostrich. He also owns an iron-headed lance, a mighty bow, a stone that smells bad when Chaos is near, and an eagle head-dress. His lance is seldom used now and he can no more string the bow. It should belong to his sons, but they were killed at Moonbroth. His successor will take it when he dies. Sun-Through-Clouds has lead the tribe for many hands of years. He has lead the tribe well, protecting the herds and stealing cows from other tribes. He stood by his principles when the Sable Nation decided to join the Red Man's army, although it was an unpopular decision. He was forced to head out into the wastes with only half a clan; the other half decided to stay and join other Sable tribes. Normally, Sun-Through-Clouds says little in tribal council meetings. He is happy to let Queen Many-Feet make the decisions about the day-to-day running of the tribe, according to the Covenant of Waha. He only interjects when he feels his role as protector is necessary. During the men's meetings, he also speaks little, preferring to allow the younger men to make speeches and put forward alternative courses of action. He allows them to argue until feelings either run too high or the argument is finished. When he does speak, everyone listens as he gives wise words and acts to balance the factions in the tribe, to preserve what little unity there is left. He is expected to die soon. Because of this, and because he has no sons to succeed him, many braves are vying for position to be the next Khan. They are split into two factions: the pro-Lunars, led by Queen Many-Feet (with the brave Hyena-Tail as her spokesman) and the anti-Lunars, led by Little Mouse, the Storm Khan. At the moment, the pro-Lunars are ascendant within the clan as they have persuaded Sun-Through-Clouds to return to Prax. Many braves are allying themselves to one or other of these factions. Sun-Through-Clouds expects the clan to split again when he dies, and he prays that he can survive and keep the clan together until it can regain strength in Prax. === Rani Many-Feet === (Wilful, protective, manipulative) This woman is the herd-mother, comforter, healer and guide of the people. She is skilled in the lore of Sables and can find water and fodder in even trackless desert. However, it is often that the oases are occupied by other, stronger clans. Her sole passion is for the health and safety of the herds and she will endure any hardship and undergo any trial to preserve them. She feels that she is the best able to lead the clan and that its destiny lies back in Prax, no longer an outcast from the Sable Nation. She has persuaded Sun-Through-Clouds to lead the clan back there and is encouraging support within the clan for her plan to join the Lunar sympathisers. === Nine Sticks, Speaker-to-Spirits === (Deranged, Delusional, Impertinent) This shaman is the clan's speaker to spirits and its ancestors. He deliberately sets himself apart from the remainder of the clan, preferring the company of the inhabitants of the Other Side to his own kind. To show his separation, he dresses in the clothes of a woman. He fought at Moonbroth, and only survived the Lunar treachery because he know the secrets of Inora, the White Princess. === Limping Bull, Storm Bull Berserk === (Fanatical, arrogant, violent) He leads the Running Dogs society, open to only those men that have killed Chaos; there are three members at the moment. This society also acts as 'police' in the clan. There are moves to expand the membership of this society, following the massacre of the clan's Storm Bullers. === Bat's Eye === (Eloquent, companiable, scathing) This ageing brave acts as Khan Sun-through-Clouds' eyes and ears throughout the clan. People don't mind this, as he is always friendly and his tipi is always filled with people eager to hear is stories of far-away lands and strange people. He often uses these occasions to feel the mood of the men, and uses his quick tongue to chastise those that have not acted like a true warrior. Due to his way with words, he is also the clan's herald. He is the one that goes out to meet and parley with other clans when we meet. === Hyena-Tail === (Weak-willed, frustrated, eloquent) This brave is the mouthpiece of Queen Many-Feet in the men's council. He is despised for being so weak-willed as to be lead by a woman, but many people agree with what he says. He is not a great warrior, but is a competent hunter and has captured some enemies' beasts. == Distinguishing Clan Features == * ''Animals'': yellow-stained tail (the yellow die is obtained from boiling down Sable gall-bladders). * ''Men'': yellow feathers in headband, zigzag tattoos on cheeks. * ''Women'': twice-braided hair, yellow stripe down the back of tunics. == Spirits worshipped == {| ! Spirit !! Highest ranking member !! Other high-lvl !! Other initiates !! Lay members !! Temple size |- |Waha || Sun-Through-Clouds (Khan) || 1 || 15 || 5 || Small |- |Erithra || Many Feet (Rani) || 2 || 24 || 10 || Small |- |Daka Fal || Nine Sticks (Shaman Priest) || 1 || 1 || 3 || Shrine |- |Storm Bull || Limping Bull (Initiate) || 0 || 0 || 2 || None* |- |Foundchild || Fallen Branch (Initiate) || 0 || 4 || 5 || Shrine |- |Chalana || Flowers' Scent (Acolyte) || 0 || 2 || 4 || Shrine |- |Issaries || Bat's Eye (Acolyte) || 0 || 1 || 2 || None |- |Lankoring || Small Hawk (Lay member) || 0 || 0 || 0 || None |- |Humakt || Antak-Dor-Tak (?) || 0 || 0 || 1 || None |- |Eurmal || Trail's End (Initiate) || 0 || 0 || 13 || None |} * The shrine to Storm Bull is temproarily inactive, due to a near-zero congregation. == Demographics of the Yellowtail Clan == {| ! Ages !! Male !! Female !! Total |- |0-1 || 1 || 1 || 2 |- |1-5 || 3 || 3 || 6 |- |5-10 || 3 || 3 || 6 |- |10-15 || 2 || 3 || 5 |- |15-20 || 2 || 3 || 5 |- |20-25 || 2 || 3 || 5 |- |25-30 || 2 || 3 || 5 |- |30-35 || 2 || 2 || 4 |- |35-40 || 1 || 2 || 3 |- |40-45 || 1 || 2 || 3 |- |45-50 || 1 || 1 || 2 |- |50-55 || 1 || 1 || 2 |- |55+ || 1 || 1 || 2 |- |Totals || 22 || 28 || 50 |} Plus 6 female slaves aged 20-30 === Herd size === {| ! !! Current Herd !! Required Herd |- |Herd size || 400 || (560) |- |Males || 40 || (56) |- |Females || 360 || (504) |- |Males born || 61 || (120.20) |- |Females born || 54.1 || (106.60) |- |Male calves dying || 42.7 || (60.10) |- |Female calves dying || 37.9 || (53.30) |- |Male adults dying || 4 || (5.6) |- |Female adults dying || 36 || (50.4) |- |Dead animals (adult equiv) || 80.3 || (112.70) |- |Net herd gain || -5.4 || (57.4) |- |Females giving milk || 208.8 || (226.8) |} You can read the [[history of the Yellowtail Clan]]. [[Category:Yellowtail]] 2328e160f179a643ddf0662826d0e72e74c391cf 0 1474 1610 2007-11-23T18:32:11Z Neil 2 wikitext text/x-wiki A quick guide to security on Windows and clearing up a malware infestation. [http://www.theregister.co.uk The Register] has a [http://www.theregister.co.uk/2004/11/11/av_workshop/ guide to sanitising your PC], which basically boils down to using the [http://www.safer-networking.org/en/index.html Spyboy search-and-destroy] program. [http://www.codinghorror.com Coding Horror] has a [http://www.codinghorror.com/blog/archives/000888.html guide to doing it manually] (more time consuming but probably more complete). If you want a firewall, [http://www.zonealarm.com/store/content/company/products/znalm/freeDownload.jsp ZoneAlarm] is about the best one out there, and it's free. To prevent further attacks, make sure that users don't have Administrator privileges. First, set the Admin password to something you'll remember. Then move all the users on the machine to the 'User' group, not the 'Admin' group. When you want something to run with administrator rights, right-click on the program and select 'Run As...'. Read [http://kb.iu.edu/data/anro.html more instructions on setting the Admin password]. If you forget it, here's some [http://pubs.logicalexpressions.com/pub0009/LPMArticle.asp?ID=305 instructions on how to recover the admin password]. Finally, a [http://www.windowsnetworking.com/articles_tutorials/Running-Windows-Under-Non-Admin-Accounts.html tutorial on how to run Windows under a non-admin account]. Read some [http://mcs.open.ac.uk/safecomputing/ general advice on safe computing]. da71c33de015fc390ab4b2427ef5c0d6197e2fb8 1611 1610 2007-11-23T19:27:49Z Neil 2 wikitext text/x-wiki A quick guide to security on Windows and clearing up a malware infestation. [http://www.theregister.co.uk The Register] has a [http://www.theregister.co.uk/2004/11/11/av_workshop/ guide to sanitising your PC], which basically boils down to using the [http://www.safer-networking.org/en/index.html Spyboy search-and-destroy] program. [http://www.codinghorror.com Coding Horror] has a [http://www.codinghorror.com/blog/archives/000888.html guide to doing it manually] (more time consuming but probably more complete). If you want a firewall, [http://www.zonealarm.com/store/content/company/products/znalm/freeDownload.jsp ZoneAlarm] is about the best one out there, and it's free. To prevent further attacks, make sure that users don't have Administrator privileges. First, set the Admin password to something you'll remember. Then move all the users on the machine to the 'User' group, not the 'Admin' group. When you want something to run with administrator rights, right-click on the program and select 'Run As...'. Read [http://kb.iu.edu/data/anro.html more instructions on setting the Admin password]. If you forget it, here's some [http://pubs.logicalexpressions.com/pub0009/LPMArticle.asp?ID=305 instructions on how to recover the admin password]. Finally, a [http://www.windowsnetworking.com/articles_tutorials/Running-Windows-Under-Non-Admin-Accounts.html tutorial on how to run Windows under a non-admin account]. Read some [http://mcs.open.ac.uk/safecomputing/ general advice on safe computing]. One thing is to use [http://en.www.mozilla.com/en/firefox/ Firefox] for browsing the web and [http://en.www.mozilla.com/en/thunderbird/ Thunderbird] for reading mail. Many malware writers expoit weaknesses in Internet Explorer and Outlook. If you don't use those programs, you're immune to those problems. 09a6a7ea82a4b63c4f415a96248222e9b041ff7e 1612 1611 2007-11-23T19:29:44Z Neil 2 wikitext text/x-wiki A quick guide to security on Windows and clearing up a malware infestation. [http://www.theregister.co.uk The Register] has a [http://www.theregister.co.uk/2004/11/11/av_workshop/ guide to sanitising your PC], which basically boils down to using the [http://www.safer-networking.org/en/index.html Spybot search-and-destroy] program. [http://www.codinghorror.com Coding Horror] has a [http://www.codinghorror.com/blog/archives/000888.html guide to doing it manually] (more time consuming but probably more complete). If you want a firewall, [http://www.zonealarm.com/store/content/company/products/znalm/freeDownload.jsp ZoneAlarm] is about the best one out there, and it's free. To prevent further attacks, make sure that users don't have Administrator privileges. First, set the Admin password to something you'll remember. Then move all the users on the machine to the 'User' group, not the 'Admin' group. When you want something to run with administrator rights, right-click on the program and select 'Run As...'. Read [http://kb.iu.edu/data/anro.html more instructions on setting the Admin password]. If you forget it, here's some [http://pubs.logicalexpressions.com/pub0009/LPMArticle.asp?ID=305 instructions on how to recover the admin password]. Finally, a [http://www.windowsnetworking.com/articles_tutorials/Running-Windows-Under-Non-Admin-Accounts.html tutorial on how to run Windows under a non-admin account]. Read some [http://mcs.open.ac.uk/safecomputing/ general advice on safe computing]. One thing is to use [http://en.www.mozilla.com/en/firefox/ Firefox] for browsing the web and [http://en.www.mozilla.com/en/thunderbird/ Thunderbird] for reading mail. Many malware writers expoit weaknesses in Internet Explorer and Outlook. If you don't use those programs, you're immune to those problems. 051de8b4ec0c46a1279d89f5dfcf69471a0ff271 0 1475 1613 2007-11-26T10:47:32Z Neil 2 wikitext text/x-wiki This is an extended table of contents for the [[:Category:Blue Planet|Blue Planet]] Players' Guide. {| |- | '''''Chapter 1: Welcome to Blue Planet''''' || '''''4''''' |- | &nbsp;&nbsp;'''2199AD''' || '''5''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Lesear Effect || 5 |- | &nbsp;&nbsp;'''Blue Planet v2''' || '''6''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''What is Blue Planet?'' || ''7'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Game Moderator'' || ''7'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Synergy Game System'' || ''8'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Dice || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dice Pools || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Target Numbers || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Task Rolls || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Making the Game your Own || 8 |- | &nbsp;&nbsp;'''Poseidon: A Survival Guide''' || '''9''' |- | '''''Chapter 2: Character Creation''''' || '''''25''''' |- | &nbsp;&nbsp;'''Concept''' || '''26''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Group Concept'' || ''26'' |- | &nbsp;&nbsp;'''Power Level and Game Balance''' || '''26''' |- | Box: Twenty Questions | 27 |- | &nbsp;&nbsp;'''Characteristics''' || '''28''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Attributes'' || ''28'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Derived Attributes'' || ''28'' |- | &nbsp;&nbsp;'''Character Species''' || '''28''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Pure-strain'' || ''29'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: In the Service of Justice || 29 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Modified'' || ''31'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Genetic Redesign'' || ''31'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dolphin'' || ''32'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Killer Whale'' || ''32'' |- | &nbsp;&nbsp;'''Modifying Attributes''' || '''33''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Point System'' || ''33'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Random System'' || ''33'' |- | &nbsp;&nbsp;'''Abilities''' || '''34''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Modi Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Genie Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cetacean Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Echolocation || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Environmental Sensitivity || 34 |- | &nbsp;&nbsp;'''Aptitudes''' || '''37''' |- | &nbsp;&nbsp;'''Skills''' || '''38''' |- | &nbsp;&nbsp;'''Training Packages''' || '''39''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Custom Skills'' || ''39'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Origin Packages'' || ''40'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Background Packages'' || ''41'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Professional Packages'' || ''43'' |- | &nbsp;&nbsp;'''Skill Descriptions''' || '''51''' |- | &nbsp;&nbsp;'''Sample Character: Bush Pilot''' || '''58''' |- | &nbsp;&nbsp;'''Sample Character: Dolphin Engineer''' || '''60''' |- | &nbsp;&nbsp;'''Sample Character: Field Biologist''' || '''62''' |- | &nbsp;&nbsp;'''Sample Character: Gangster''' || '''64''' |- | &nbsp;&nbsp;'''Sample Character: GEO Marshal''' || '''66''' |- | &nbsp;&nbsp;'''Sample Character: GEO Shock Trooper''' || '''68''' |- | &nbsp;&nbsp;'''Sample Character: Native Ecoterrorist''' || '''70''' |- | &nbsp;&nbsp;'''Sample Character: Orca Peacekeeper''' || '''72''' |- | &nbsp;&nbsp;'''Sample Character: Silva Warden''' || '''74''' |- | &nbsp;&nbsp;'''Character Development''' || '''76''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Earning Character Improvement Points'' || ''76'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spending Character Improvement Points'' || ''76'' |- | '''''Chapter 3: Character Profile''''' || '''''77''''' |- | &nbsp;&nbsp;'''Goal''' || '''78''' |- | &nbsp;&nbsp;'''Motivation''' || '''79''' |- | &nbsp;&nbsp;'''Attitude''' || '''80''' |- | &nbsp;&nbsp;'''Player's Choice''' || '''80''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Features'' || ''81'' |- | &nbsp;&nbsp;'''Roles''' || '''82''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Administrator'' || ''82'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artist'' || ''83'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Civilian Pilot'' || ''83'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Doctor'' || ''84'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ecoterrorist'' || ''85'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Freebooter'' || ''85'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Frontiersman'' || ''86'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Gangster'' || ''86'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Cold Soup || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Magistrate'' || ''88'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Marshall'' || ''89'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Patrol'' || ''90'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Peacekeeper'' || ''91'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Shock Trooper'' || ''91'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Incorporate Security'' || ''92'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Intelligence Agent'' || ''92'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Journalist'' || ''93'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Medic'' || ''93'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Mercenary'' || ''94'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Military Pilot'' || ''95'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Miner'' || ''96'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Healer'' || ''97'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Insurgent'' || ''98'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Patrol'' || ''98'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Sell-Out'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Opportunist'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Pioneer'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Private Investigator'' || ''100'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Prospector'' || ''100'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sailor'' || ''101'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Black Charlie || 102 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Scholar'' || ''102'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Scientist'' || ''103'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spacer'' || ''104'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Technician'' || ''104'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Thug'' || ''106'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Trader'' || ''106'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Warden'' || ''107'' |- | '''''Chapter 4: Synergy Game System''''' || '''''108''''' |- | &nbsp;&nbsp;'''Task Resolution''' || '''109''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Skilled Tasks'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aptitude'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Target Number'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Task Difficulty'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Unskilled Tasks'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Action Value'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Attribute Rolls'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Opposed Rolls'' || ''110'' |- | &nbsp;&nbsp;'''Combat''' || '''110''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Action Rounds'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Actions'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Initiative'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Multiple Actions'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Holding Actions'' || ''112'' |- | &nbsp;&nbsp;'''Ranged Combat''' || '''112''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Range'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Size'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Movement'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cover'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aiming'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Recoil'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Automatic Fire'' || ''113'' |- | &nbsp;&nbsp;'''Close Combat''' || '''114''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Manoeuvres'' || ''114'' |- | &nbsp;&nbsp;'''Situational Modifiers''' || '''116''' |- | &nbsp;&nbsp;'''Damage''' || '''116''' |- | &nbsp;&nbsp;'''Wounds''' || '''117''' |- | &nbsp;&nbsp;'''Damage Resolution''' || '''117''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Called Shots'' || ''118'' |- | &nbsp;&nbsp;'''First Aid''' || '''118''' |- | &nbsp;&nbsp;'''Special Damage''' || '''118''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Falling'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Explosives'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Burning and Electrical Shock'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hypothermia'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poison'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Suffocation and Drowning'' || ''120'' |- | &nbsp;&nbsp;'''Healing and Recovery''' || '''120''' |- | &nbsp;&nbsp;'''Vehicle Damage''' || '''120''' |- | '''''Chapter 5: Hardware''''' || '''''122''''' |- | &nbsp;&nbsp;'''Technology Key''' || '''123''' |- | &nbsp;&nbsp;'''Power Supplies''' || '''123''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Batteries'' || ''123'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fuel Cells'' || ''124'' |- | &nbsp;&nbsp;'''Electronics Components''' || '''125''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Access Chip'' || ''125'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Interactive Access Chip'' || ''125'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Interface Technology'' || ''125'' |- | &nbsp;&nbsp;'''Computers''' || '''126''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Body Computers'' || ''126'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dataspike'' || ''127'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Main Computers'' || ''127'' |- | &nbsp;&nbsp;'''Communications''' || '''127''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Personal Communications'' || ''127'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Uplink Communicators'' || ''127'' |- | &nbsp;&nbsp;'''Robotics and Remotes''' || '''128''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Robots'' || ''128'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Agricultural Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Construction Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delivery Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Domestic Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Remotes'' || ''129'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cetacean Remote || 129 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Combat Remote || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Recon Remote || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Surveillance Remote || 130 |- | &nbsp;&nbsp;'''Sensors''' || '''130''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Broad Spectrum Visor'' || ''130'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat HUD'' || ''130'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Digital Binoculars'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hand-Held Sonar Transceiver'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Security Monitors'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensor Array'' || ''131'' |- | &nbsp;&nbsp;'''Medical Gear''' || '''132''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Biomonitor'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Coagulant Pads'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cold-Sleep Capusules'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cerebral Cryo-Oxygenators'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Field Surgery Kit'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Standard Medical Kit'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ultrasonic Hypodermic'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Wound Glue'' || ''134'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Drugs'' || ''134'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Antibiotics/Antivirals || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anti-Shock || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Auntie Susie || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Healing Booster || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Pain Inhibitor || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reflex Serum || 135 |- | &nbsp;&nbsp;'''Life Support''' || '''136''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artificial Gill'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Bacterial Life Support EVA System'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Diving Mask'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Drysuit'' || ''137'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Gill Suit'' || ''137'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hard Suit'' || ''137'' |- | &nbsp;&nbsp;'''Survival Gear''' || '''138''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Filter Mask'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fire Paste'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Locator Beacon'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Portable Navigation Suite'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Rescue Ball'' || ''139'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Guide'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Grenade'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Plastic'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Purifiers'' || ''140'' |- | &nbsp;&nbsp;'''Personal Propulsion''' || '''141''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''MHD Harness'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''MHD Sled'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Manoeuvre Jets'' || ''141'' |- | &nbsp;&nbsp;'''Remote Vehicle Technology''' || '''141''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Remotely Piloted Submersibles'' || ''141'' |- | &nbsp;&nbsp;'''Cetacean Technologies''' || '''142''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artificial Gill'' || ''142'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''CICADA'' || ''142'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cetacean Weapon Harness'' || ''143'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sonic Trode'' || ''143'' |- | &nbsp;&nbsp;'''Materials''' || '''143''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Bioplastic'' || ''143'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flex Grade || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Foamed Plastic || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industrial Grade || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Standard Grade || 144 |- | &nbsp;&nbsp;'''Body Armour''' || '''144''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Armour'' || ''144'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Helmet'' || ''144'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Phototropic BDU'' || ''145'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Personal Body Armour'' || ''145'' |- | &nbsp;&nbsp;'''Melee Weapons''' || '''145''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Diamond Knife'' || ''145'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Stun Gloves and Batons'' || ''145'' |- | &nbsp;&nbsp;'''Firearms''' || '''146''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Handguns'' || ''146'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Concealable || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Large Calibre || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Small Calibre || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Stunguns'' || ''147'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Longarms'' || ''148'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hunting Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Shotgun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sniper Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spear Gun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Submachine Gun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Heavy Weapons'' || ''149'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Cannon || 149 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Autocannon || 149 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Grenade Launcher || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Heavy Machinegun || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Light Machinegun || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Torpedo Cannon || 150 |- | &nbsp;&nbsp;'''Firearm Accessories''' || '''151''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ammunition'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour-Piercing Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Explosive Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flechette Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Standard Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Suppression Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Assault Harness'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Laser Designator'' || ''152'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Optics'' || ''152'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Electronic Scopes || 152 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Optical Scopes || 152 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Targeting Interlink'' || ''152'' |- | &nbsp;&nbsp;'''Grenades''' || '''153''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Destructive Grenades'' || ''153'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour-Piercing Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Concussion Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Fragmentation Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Incendiary Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Suppression Grenades'' || ''153'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Adhesive Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flash-Bang || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nausea Grenade || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sleep Grenade || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Smoke Grenade || 153 |- | &nbsp;&nbsp;'''Vehicles''' || '''154''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Motorcycle'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hovercraft'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;APC Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Heavy Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Light Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Jumpcraft'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Jumpcraft || 156 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cargo Jumpcraft || 156 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Jumpbike || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Patrol Jumpcraft || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility Jumpcraft || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;''VTOL Aircraft'' || ''157'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility VTOL || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VTOL Strike Fighter || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Submersibles'' || ''158'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cetacean Power Shell || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Submersible Interceptor || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Research Submersible || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility Submersible || 160 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Watercraft'' || ''160'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hydrofoil || 160 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Power Ski || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sailboats || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Vehicle Accessories'' || ''161'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Auxiliary Fuel Tank || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Computer-Assisted Targeting System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ECM Suite || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Ejection System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Evac Pod || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Gunner's Turret || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Onboard Computer || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Radar Suite || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Security System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sonar Suite || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Targeting Computer || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VR Cockpit || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Guided Weapons'' || ''163'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infantry Missile Launcher || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Multipurpose Anti-Vehicle Missile || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Torpedo || 163 |- | '''''Chapter 6: Biotech''''' || '''''164''''' |- | &nbsp;&nbsp;'''Biotechnology''' || '''165''' |- | &nbsp;&nbsp;'''Cybernetic Modification''' || '''165''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Implant Computers'' || ''166'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Calculator || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Microcomputer || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Sensory Recorder || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Translator || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Neural Jack'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Pain Inhibitor'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Programmed Reflexes'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Programs || 168 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensor and Technical Cyberwear'' || ''168'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrared Sensors || 168 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Radar || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Micro-Toolkit || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spyware || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Anatomical Cyberwear'' || ''169'' |- | &nbsp;&nbsp;'''Genetic Redesign''' || '''170''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aquaforms'' || ''170'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hybrids'' || ''171'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spacers'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Transhumans'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Genies and Biomods'' || ''173'' |- | &nbsp;&nbsp;'''Genetic Modification''' || '''173''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Medical Biomods'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Longevity Therapy || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Regeneration || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Metabolic Biomods'' || ''174'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anti-Poison || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Improved Blood Oxygenation || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Mind-Job || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Respiratory Filter || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Multiglands || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Salt Tolerance || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Implanted Biomods'' || ''175'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Electro-Muscular Analogues || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Immunological Symbiote || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensory Biomods'' || ''176'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Amplified Hearing || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Echolocation || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Enhanced Smell || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Extra Sensory Perception Enhancement || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Telescopic Vision || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Full-Body Biomods'' || ''177'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Accelerated Neurons || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Aquaform || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Body Sculpting || 178 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GEO Shock Trooper || 178 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hybrid || 179 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spacer || 179 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Transhuman || 179 |- | '''''Chapter 7: Future History''''' | '''''180''''' |- | &nbsp;&nbsp;'''The Athena Project''' || '''181''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Prometheus II Returns || 184 |- | &nbsp;&nbsp;'''The Blight''' || '''184''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Europe'' || ''185'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Americas'' || ''187'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Africa and the Middle East'' || ''188'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Asia'' || ''189'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Antartica'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Earth Orbit'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Luna, Mars Colony, and the Asteroid Belt'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Death Toll'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Global Ecology Organization'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Shooting the Hole || 192 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GEO Protectorates and Independents || 192 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Incorporate States'' || ''193'' |- | &nbsp;&nbsp;'''The Natives and the Abandonment''' || '''196''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: The Cooler || 198 |- | &nbsp;&nbsp;'''Recontact''' || '''199''' |- | &nbsp;&nbsp;'''The Long John Rush''' || '''200''' |- | &nbsp;&nbsp;'''On the Horizon''' || '''201''' |- | '''''Chapter 8: On the Frontier''''' || '''''202''''' |- | &nbsp;&nbsp;'''A World in Conflict''' || '''203''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Natives'' || ''203'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture and Psychology || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Native View || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Relations between Diver and Squid || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Natives and Poseidon || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Natives and Earth || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Technology || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Native Settlements || 208 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Aquaculture and Agriculture || 209 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Lifestyle || 210 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Ritual and Ceremony || 211 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The GEO on Poseidon'' || ''212'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Human Resources || 212 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Natural Resources || 212 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Science and Technology || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Communications || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Internal Security || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State and Internal Affairs || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armed Forces || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Justice || 214 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Trade and Industry || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Incorporate States on Poseidon'' || ''216'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anasi Systems || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Atlas Materials || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Biogene || 217 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Long John || 217 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dundalk Shipbuilding || 218 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Gendiver || 218 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hanover Industries || 220 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hydrospan || 221 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Lavender Organics || 222 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MacLeod Enforcement || 223 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nippon Industrial State || 224 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Newcomers'' || ''224'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Pioneers || 225 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Colonial Calendar || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Prospectors || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Opportunists || 227 |- | &nbsp;&nbsp;'''Cetaceans''' || '''227''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 227 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Life on Poseidon || 228 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dolphins'' || ''229'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anatomy and Physiology || 229 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Psychology || 229 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sociology || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture || 231 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Killer Whales'' || ''232'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anatomy and Physiology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Psychology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sociology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture || 233 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Technology'' || ''233'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Lifestyle'' || ''234'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Transportation'' || ''234'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Communication'' || ''234'' |- | '''''Chapter 9: Timeline''''' || '''''236''''' |} [[Category:Blue Planet]] e97a6444c4f2086308feaa80059ac235691a1848 1616 1613 2007-11-26T11:13:42Z Neil 2 wikitext text/x-wiki This is an extended table of contents for the [[:Category:Blue Planet|Blue Planet]] Players' Guide. {| |- | '''''Chapter 1: Welcome to Blue Planet''''' || '''''4''''' |- | &nbsp;&nbsp;'''2199AD''' || '''5''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Lesear Effect || 5 |- | &nbsp;&nbsp;'''Blue Planet v2''' || '''6''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''What is Blue Planet?'' || ''7'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Game Moderator'' || ''7'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Synergy Game System'' || ''8'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Dice || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dice Pools || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Target Numbers || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Task Rolls || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Making the Game your Own || 8 |- | &nbsp;&nbsp;'''Poseidon: A Survival Guide''' || '''9''' |- | '''''Chapter 2: Character Creation''''' || '''''25''''' |- | &nbsp;&nbsp;'''Concept''' || '''26''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Group Concept'' || ''26'' |- | &nbsp;&nbsp;'''Power Level and Game Balance''' || '''26''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Twenty Questions || 27 |- | &nbsp;&nbsp;'''Characteristics''' || '''28''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Attributes'' || ''28'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Derived Attributes'' || ''28'' |- | &nbsp;&nbsp;'''Character Species''' || '''28''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Pure-strain'' || ''29'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: In the Service of Justice || 29 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Modified'' || ''31'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Genetic Redesign'' || ''31'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dolphin'' || ''32'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Killer Whale'' || ''32'' |- | &nbsp;&nbsp;'''Modifying Attributes''' || '''33''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Point System'' || ''33'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Random System'' || ''33'' |- | &nbsp;&nbsp;'''Abilities''' || '''34''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Modi Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Genie Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cetacean Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Echolocation || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Environmental Sensitivity || 34 |- | &nbsp;&nbsp;'''Aptitudes''' || '''37''' |- | &nbsp;&nbsp;'''Skills''' || '''38''' |- | &nbsp;&nbsp;'''Training Packages''' || '''39''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Custom Skills'' || ''39'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Origin Packages'' || ''40'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Background Packages'' || ''41'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Professional Packages'' || ''43'' |- | &nbsp;&nbsp;'''Skill Descriptions''' || '''51''' |- | &nbsp;&nbsp;'''Sample Character: Bush Pilot''' || '''58''' |- | &nbsp;&nbsp;'''Sample Character: Dolphin Engineer''' || '''60''' |- | &nbsp;&nbsp;'''Sample Character: Field Biologist''' || '''62''' |- | &nbsp;&nbsp;'''Sample Character: Gangster''' || '''64''' |- | &nbsp;&nbsp;'''Sample Character: GEO Marshal''' || '''66''' |- | &nbsp;&nbsp;'''Sample Character: GEO Shock Trooper''' || '''68''' |- | &nbsp;&nbsp;'''Sample Character: Native Ecoterrorist''' || '''70''' |- | &nbsp;&nbsp;'''Sample Character: Orca Peacekeeper''' || '''72''' |- | &nbsp;&nbsp;'''Sample Character: Silva Warden''' || '''74''' |- | &nbsp;&nbsp;'''Character Development''' || '''76''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Earning Character Improvement Points'' || ''76'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spending Character Improvement Points'' || ''76'' |- | '''''Chapter 3: Character Profile''''' || '''''77''''' |- | &nbsp;&nbsp;'''Goal''' || '''78''' |- | &nbsp;&nbsp;'''Motivation''' || '''79''' |- | &nbsp;&nbsp;'''Attitude''' || '''80''' |- | &nbsp;&nbsp;'''Player's Choice''' || '''80''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Features'' || ''81'' |- | &nbsp;&nbsp;'''Roles''' || '''82''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Administrator'' || ''82'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artist'' || ''83'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Civilian Pilot'' || ''83'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Doctor'' || ''84'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ecoterrorist'' || ''85'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Freebooter'' || ''85'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Frontiersman'' || ''86'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Gangster'' || ''86'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Cold Soup || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Magistrate'' || ''88'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Marshall'' || ''89'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Patrol'' || ''90'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Peacekeeper'' || ''91'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Shock Trooper'' || ''91'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Incorporate Security'' || ''92'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Intelligence Agent'' || ''92'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Journalist'' || ''93'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Medic'' || ''93'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Mercenary'' || ''94'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Military Pilot'' || ''95'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Miner'' || ''96'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Healer'' || ''97'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Insurgent'' || ''98'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Patrol'' || ''98'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Sell-Out'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Opportunist'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Pioneer'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Private Investigator'' || ''100'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Prospector'' || ''100'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sailor'' || ''101'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Black Charlie || 102 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Scholar'' || ''102'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Scientist'' || ''103'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spacer'' || ''104'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Technician'' || ''104'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Thug'' || ''106'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Trader'' || ''106'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Warden'' || ''107'' |- | '''''Chapter 4: Synergy Game System''''' || '''''108''''' |- | &nbsp;&nbsp;'''Task Resolution''' || '''109''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Skilled Tasks'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aptitude'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Target Number'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Task Difficulty'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Unskilled Tasks'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Action Value'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Attribute Rolls'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Opposed Rolls'' || ''110'' |- | &nbsp;&nbsp;'''Combat''' || '''110''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Action Rounds'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Actions'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Initiative'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Multiple Actions'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Holding Actions'' || ''112'' |- | &nbsp;&nbsp;'''Ranged Combat''' || '''112''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Range'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Size'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Movement'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cover'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aiming'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Recoil'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Automatic Fire'' || ''113'' |- | &nbsp;&nbsp;'''Close Combat''' || '''114''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Manoeuvres'' || ''114'' |- | &nbsp;&nbsp;'''Situational Modifiers''' || '''116''' |- | &nbsp;&nbsp;'''Damage''' || '''116''' |- | &nbsp;&nbsp;'''Wounds''' || '''117''' |- | &nbsp;&nbsp;'''Damage Resolution''' || '''117''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Called Shots'' || ''118'' |- | &nbsp;&nbsp;'''First Aid''' || '''118''' |- | &nbsp;&nbsp;'''Special Damage''' || '''118''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Falling'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Explosives'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Burning and Electrical Shock'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hypothermia'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poison'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Suffocation and Drowning'' || ''120'' |- | &nbsp;&nbsp;'''Healing and Recovery''' || '''120''' |- | &nbsp;&nbsp;'''Vehicle Damage''' || '''120''' |- | '''''Chapter 5: Hardware''''' || '''''122''''' |- | &nbsp;&nbsp;'''Technology Key''' || '''123''' |- | &nbsp;&nbsp;'''Power Supplies''' || '''123''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Batteries'' || ''123'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fuel Cells'' || ''124'' |- | &nbsp;&nbsp;'''Electronics Components''' || '''125''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Access Chip'' || ''125'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Interactive Access Chip'' || ''125'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Interface Technology'' || ''125'' |- | &nbsp;&nbsp;'''Computers''' || '''126''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Body Computers'' || ''126'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dataspike'' || ''127'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Main Computers'' || ''127'' |- | &nbsp;&nbsp;'''Communications''' || '''127''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Personal Communications'' || ''127'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Uplink Communicators'' || ''127'' |- | &nbsp;&nbsp;'''Robotics and Remotes''' || '''128''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Robots'' || ''128'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Agricultural Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Construction Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delivery Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Domestic Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Remotes'' || ''129'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cetacean Remote || 129 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Combat Remote || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Recon Remote || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Surveillance Remote || 130 |- | &nbsp;&nbsp;'''Sensors''' || '''130''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Broad Spectrum Visor'' || ''130'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat HUD'' || ''130'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Digital Binoculars'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hand-Held Sonar Transceiver'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Security Monitors'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensor Array'' || ''131'' |- | &nbsp;&nbsp;'''Medical Gear''' || '''132''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Biomonitor'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Coagulant Pads'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cold-Sleep Capusules'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cerebral Cryo-Oxygenators'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Field Surgery Kit'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Standard Medical Kit'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ultrasonic Hypodermic'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Wound Glue'' || ''134'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Drugs'' || ''134'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Antibiotics/Antivirals || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anti-Shock || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Auntie Susie || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Healing Booster || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Pain Inhibitor || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reflex Serum || 135 |- | &nbsp;&nbsp;'''Life Support''' || '''136''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artificial Gill'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Bacterial Life Support EVA System'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Diving Mask'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Drysuit'' || ''137'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Gill Suit'' || ''137'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hard Suit'' || ''137'' |- | &nbsp;&nbsp;'''Survival Gear''' || '''138''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Filter Mask'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fire Paste'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Locator Beacon'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Portable Navigation Suite'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Rescue Ball'' || ''139'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Guide'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Grenade'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Plastic'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Purifiers'' || ''140'' |- | &nbsp;&nbsp;'''Personal Propulsion''' || '''141''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''MHD Harness'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''MHD Sled'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Manoeuvre Jets'' || ''141'' |- | &nbsp;&nbsp;'''Remote Vehicle Technology''' || '''141''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Remotely Piloted Submersibles'' || ''141'' |- | &nbsp;&nbsp;'''Cetacean Technologies''' || '''142''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artificial Gill'' || ''142'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''CICADA'' || ''142'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cetacean Weapon Harness'' || ''143'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sonic Trode'' || ''143'' |- | &nbsp;&nbsp;'''Materials''' || '''143''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Bioplastic'' || ''143'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flex Grade || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Foamed Plastic || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industrial Grade || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Standard Grade || 144 |- | &nbsp;&nbsp;'''Body Armour''' || '''144''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Armour'' || ''144'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Helmet'' || ''144'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Phototropic BDU'' || ''145'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Personal Body Armour'' || ''145'' |- | &nbsp;&nbsp;'''Melee Weapons''' || '''145''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Diamond Knife'' || ''145'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Stun Gloves and Batons'' || ''145'' |- | &nbsp;&nbsp;'''Firearms''' || '''146''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Handguns'' || ''146'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Concealable || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Large Calibre || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Small Calibre || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Stunguns'' || ''147'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Longarms'' || ''148'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hunting Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Shotgun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sniper Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spear Gun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Submachine Gun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Heavy Weapons'' || ''149'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Cannon || 149 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Autocannon || 149 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Grenade Launcher || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Heavy Machinegun || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Light Machinegun || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Torpedo Cannon || 150 |- | &nbsp;&nbsp;'''Firearm Accessories''' || '''151''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ammunition'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour-Piercing Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Explosive Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flechette Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Standard Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Suppression Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Assault Harness'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Laser Designator'' || ''152'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Optics'' || ''152'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Electronic Scopes || 152 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Optical Scopes || 152 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Targeting Interlink'' || ''152'' |- | &nbsp;&nbsp;'''Grenades''' || '''153''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Destructive Grenades'' || ''153'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour-Piercing Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Concussion Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Fragmentation Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Incendiary Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Suppression Grenades'' || ''153'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Adhesive Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flash-Bang || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nausea Grenade || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sleep Grenade || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Smoke Grenade || 153 |- | &nbsp;&nbsp;'''Vehicles''' || '''154''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Motorcycle'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hovercraft'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;APC Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Heavy Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Light Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Jumpcraft'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Jumpcraft || 156 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cargo Jumpcraft || 156 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Jumpbike || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Patrol Jumpcraft || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility Jumpcraft || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;''VTOL Aircraft'' || ''157'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility VTOL || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VTOL Strike Fighter || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Submersibles'' || ''158'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cetacean Power Shell || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Submersible Interceptor || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Research Submersible || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility Submersible || 160 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Watercraft'' || ''160'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hydrofoil || 160 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Power Ski || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sailboats || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Vehicle Accessories'' || ''161'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Auxiliary Fuel Tank || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Computer-Assisted Targeting System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ECM Suite || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Ejection System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Evac Pod || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Gunner's Turret || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Onboard Computer || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Radar Suite || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Security System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sonar Suite || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Targeting Computer || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VR Cockpit || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Guided Weapons'' || ''163'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infantry Missile Launcher || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Multipurpose Anti-Vehicle Missile || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Torpedo || 163 |- | '''''Chapter 6: Biotech''''' || '''''164''''' |- | &nbsp;&nbsp;'''Biotechnology''' || '''165''' |- | &nbsp;&nbsp;'''Cybernetic Modification''' || '''165''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Implant Computers'' || ''166'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Calculator || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Microcomputer || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Sensory Recorder || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Translator || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Neural Jack'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Pain Inhibitor'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Programmed Reflexes'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Programs || 168 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensor and Technical Cyberwear'' || ''168'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrared Sensors || 168 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Radar || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Micro-Toolkit || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spyware || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Anatomical Cyberwear'' || ''169'' |- | &nbsp;&nbsp;'''Genetic Redesign''' || '''170''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aquaforms'' || ''170'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hybrids'' || ''171'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spacers'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Transhumans'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Genies and Biomods'' || ''173'' |- | &nbsp;&nbsp;'''Genetic Modification''' || '''173''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Medical Biomods'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Longevity Therapy || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Regeneration || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Metabolic Biomods'' || ''174'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anti-Poison || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Improved Blood Oxygenation || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Mind-Job || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Respiratory Filter || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Multiglands || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Salt Tolerance || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Implanted Biomods'' || ''175'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Electro-Muscular Analogues || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Immunological Symbiote || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensory Biomods'' || ''176'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Amplified Hearing || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Echolocation || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Enhanced Smell || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Extra Sensory Perception Enhancement || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Telescopic Vision || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Full-Body Biomods'' || ''177'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Accelerated Neurons || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Aquaform || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Body Sculpting || 178 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GEO Shock Trooper || 178 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hybrid || 179 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spacer || 179 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Transhuman || 179 |- | '''''Chapter 7: Future History''''' | '''''180''''' |- | &nbsp;&nbsp;'''The Athena Project''' || '''181''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Prometheus II Returns || 184 |- | &nbsp;&nbsp;'''The Blight''' || '''184''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Europe'' || ''185'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Americas'' || ''187'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Africa and the Middle East'' || ''188'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Asia'' || ''189'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Antartica'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Earth Orbit'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Luna, Mars Colony, and the Asteroid Belt'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Death Toll'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Global Ecology Organization'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Shooting the Hole || 192 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GEO Protectorates and Independents || 192 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Incorporate States'' || ''193'' |- | &nbsp;&nbsp;'''The Natives and the Abandonment''' || '''196''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: The Cooler || 198 |- | &nbsp;&nbsp;'''Recontact''' || '''199''' |- | &nbsp;&nbsp;'''The Long John Rush''' || '''200''' |- | &nbsp;&nbsp;'''On the Horizon''' || '''201''' |- | '''''Chapter 8: On the Frontier''''' || '''''202''''' |- | &nbsp;&nbsp;'''A World in Conflict''' || '''203''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Natives'' || ''203'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture and Psychology || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Native View || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Relations between Diver and Squid || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Natives and Poseidon || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Natives and Earth || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Technology || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Native Settlements || 208 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Aquaculture and Agriculture || 209 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Lifestyle || 210 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Ritual and Ceremony || 211 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The GEO on Poseidon'' || ''212'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Human Resources || 212 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Natural Resources || 212 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Science and Technology || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Communications || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Internal Security || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State and Internal Affairs || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armed Forces || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Justice || 214 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Trade and Industry || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Incorporate States on Poseidon'' || ''216'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anasi Systems || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Atlas Materials || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Biogene || 217 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Long John || 217 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dundalk Shipbuilding || 218 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Gendiver || 218 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hanover Industries || 220 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hydrospan || 221 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Lavender Organics || 222 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MacLeod Enforcement || 223 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nippon Industrial State || 224 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Newcomers'' || ''224'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Pioneers || 225 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Colonial Calendar || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Prospectors || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Opportunists || 227 |- | &nbsp;&nbsp;'''Cetaceans''' || '''227''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 227 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Life on Poseidon || 228 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dolphins'' || ''229'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anatomy and Physiology || 229 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Psychology || 229 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sociology || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture || 231 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Killer Whales'' || ''232'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anatomy and Physiology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Psychology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sociology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture || 233 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Technology'' || ''233'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Lifestyle'' || ''234'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Transportation'' || ''234'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Communication'' || ''234'' |- | '''''Chapter 9: Timeline''''' || '''''236''''' |} [[Category:Blue Planet]] c3511ba03f4432b7ae80fa1ba666c7456465b4cb 0 1476 1614 2007-11-26T11:00:18Z Neil 2 wikitext text/x-wiki This is an extended table of contents for the [[:Category:Blue Planet|Blue Planet]] Moderators' Guide. {| |- | '''''Foreword: The Moderator's Role''''' || '''''3''''' |- | &nbsp;&nbsp;'''Easter''' || '''3''' |- | &nbsp;&nbsp;'''The Moderator's Role''' || '''4''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Designing Campaigns'' || ''4'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Campaign Concept || 4 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Scenario Ideas || 6 |- | &nbsp;&nbsp;'''The Moderator's Objective''' || '''7''' |- | '''''Chapter 1: The New Frontier''''' || '''''8''''' |- | &nbsp;&nbsp;'''The Serpentis System''' || '''9''' |- | &nbsp;&nbsp;'''The Pacifica Archipelago''' || '''13''' |- | &nbsp;&nbsp;'''Haven Cluster''' || '''15''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Haven'' || ''18'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 18 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 18 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 19 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Districts of Haven || 19 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Jason and Darwin || 20 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 22 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 22 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 23 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 24 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 24 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 25 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Second Try'' || ''26'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 26 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 26 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 28 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Districts of Second Try || 29 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 30 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 30 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 30 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 30 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 31 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 32 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Lebensraum'' || ''32'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 32 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 33 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 33 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 33 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Nomad'' || ''35'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 35 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 35 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 36 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 36 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Depth Perception || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Circumstance'' || ''38'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 38 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 38 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 39 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 39 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 39 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 40 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Wall'' || ''40'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''42'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Milo Franklin || 42 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Vendor || 43 |- | &nbsp;&nbsp;'''New Hawaii''' || '''45''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Simushir'' || ''47'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 47 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 47 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 49 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 49 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 50 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 50 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 50 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 51 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 52 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Atlantis'' || ''52'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 52 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 52 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Underground || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Coronado Station'' || ''55'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 55 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 55 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 55 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 55 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Members Only || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''56'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Brian Steeg || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Rachel Lightfoot || 57 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 58 |- | &nbsp;&nbsp;'''Northwest Territories''' || '''61''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 63 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Sierra Nueva Cluster'' || ''63'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 63 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 63 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 64 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 64 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 64 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 65 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 65 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Native Unrest || 65 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 66 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Baffin Island Settlement'' || ''66'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 66 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 66 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Warpod || 68 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 70 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 73 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Santa Elena'' || ''74'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 74 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 74 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 74 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Crusoe Island Military Base'' || ''77'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 77 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 77 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 79 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Sierra Neuva War'' || ''79'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 82 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''83'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Prophet || 83 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Stuart Parsons || 83 |- | &nbsp;&nbsp;'''Prime Meridian''' || '''85''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Al-Mamlakah'' || ''87'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 89 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 89 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 90 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 90 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 91 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 92 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Undersea Habitat-2'' || ''93'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 95 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Under Pressure || 96 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 96 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Alderberg'' || ''97'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 97 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 97 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 97 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 97 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 98 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 98 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 98 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 98 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Kansas'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 99 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 99 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 99 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 100 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 100 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 100 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 100 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 101 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Islamic Faithful'' || ''101'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Meridian Frontier'' || ''102'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Bryon's Spur || 102 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Maglev Express || 102 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Stockyards || 103 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 103 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''104'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Habib Hussein Al-Muhammadi || 104 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Simon Malmoneeds || 105 |- | &nbsp;&nbsp;'''Westscape''' || '''107''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dyfedd'' || ''108'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 108 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 108 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 110 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Districts of Dyfedd || 111 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 112 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 113 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 113 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 113 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hugo D-4 || 113 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 114 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 115 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Legacy of Recontact'' || ''116'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perdition || 116 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Landgrab || 117 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Hanover Presence || 118 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Uncertain Future || 119 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Brush War || 119 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 120 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;B&ouml;se Strand || 120 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 121 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''122'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Johnny Cecily || 122 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Bernardo Oloveira || 122 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Philip Jamhuri || 123 |- | &nbsp;&nbsp;'''Zion Islands''' || '''124''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 126 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Kingston'' || ''127'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 127 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 127 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 129 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Parishes of Kingston || 129 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 131 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 131 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 131 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 132 |- | &nbsp;&nbsp;&nbsp;&nbsp;''New Fremantle'' || ''134'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 136 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 136 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fort Pacifica'' || ''137'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 137 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 137 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 137 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 137 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 138 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 138 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 138 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 138 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Nothing Ventured'' || ''139'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Hazards Casino || 139 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dragon Boy Parlour || 140 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Kingston Saloons || 140 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Bright Savanna'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Roberto &ldquo;Sugar&rdquo; McKay || 141 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Geronimo Pacheco || 142 |- | &nbsp;&nbsp;'''A View from Orbit''' || '''143''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Prosperity Station'' || ''143'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Shannon Marie || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Space Dock || 145 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Worldscape || 145 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 145 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon's Moons'' || ''146'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Proteus || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nereus || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Serpentis Belt'' || ''146'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 147 |- | '''''Chapter 2: Beyond the Frontier''''' || '''''148''''' |- | &nbsp;&nbsp;'''Oceanography for Gamers''' || '''149''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Pressure'' || ''149'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Breathing Underwater'' || ''149'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Light Underwater'' || ''150'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sound Underwater'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Temperature'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Salinity'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Tides'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Currents'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Waves'' || ''152'' |- | &nbsp;&nbsp;'''Poseidon Ecosystems''' || '''152''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Canyonlands'' || ''152'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Mangrove'' || ''153'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sargassum Islands'' || ''154'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Thermal Oases'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Tidal Mud Reefs'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Fever || 156 |- | &nbsp;&nbsp;'''The Forecast''' || '''157''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''General Climate'' || ''157'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cyclonic Storms'' || ''158'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GEO Metwatch Force Ratings || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cyclonics in the Game || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Storm Season || 160 |- | &nbsp;&nbsp;'''Poseidon Field Guide''' || '''163''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Am-Bush'' || ''163'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Blimp'' || ''164'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Carniflora'' || ''165'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Chain Beetle'' || ''166'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Digger Crab'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Eel Dragon'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fast Fungus'' || ''168'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fish'' || ''169'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fisherman'' || ''170'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ghoster'' || ''170'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Greater White'' || ''171'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hangin' Joe'' || ''172'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Harvester Worm'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hatchlings'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hexa Boar'' || ''174'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Howell's Leech'' || ''175'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Land Lizard'' || ''175'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Loggerhead'' || ''176'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Marsh Devil'' || ''177'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Needle Bush'' || ''177'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Needle Shell'' || ''178'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Night Crawler'' || ''179'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Nooniebird'' || ''179'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 180 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Polypod'' || ''180'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Kelp'' || ''181'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Mangrove'' || ''182'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Scorpion'' || ''182'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Trilobyte'' || ''183'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Reefworm'' || ''184'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Rubber Shrimp'' || ''184'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Rumble Bee'' || ''185'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Schooler'' || ''185'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Seaweaver'' || ''186'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spurts'' || ''187'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Squealers, Stick Monkeys'' || ''188'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Stone Snake'' || ''188'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sunburst'' || ''189'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Trident Fish'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Walkabout'' || ''191'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Dart'' || ''192'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Hemp'' || ''192'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Rat'' || ''193'' |- | '''''Chapter 3: Alien Legacy''''' || '''''194''''' |- | &nbsp;&nbsp;'''The Aborigines''' || '''196''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creator Legacy'' || ''196'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aboriginal Culture'' || ''196'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Technology'' || ''196'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Long John'' || ''197'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Anatomy and Physiology'' || ''197'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Morphology and Locomotion || 197 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nervous System || 198 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Senses || 198 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Metabolism || 198 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Electrochemical Abilities || 198 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reproduction || 199 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aborigine Castes'' || 199 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Breeders || 199 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Technicians || 199 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Specialists || 200 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Herders || 200 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Protectors || 200 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aborigines and the Human Invasion'' || ''201'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creator Caches'' || ''201'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Power Sources'' || ''203'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creator Facilities'' || ''203'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Machine Farm || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Storage Cache || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Template Library || 204 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hibernation Dormitory || 204 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Research Sites || 204 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creator Technology'' || ''204'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nanites || 204 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Biomechanical Drone || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Memory Ring || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Repair Bath || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Raw Materials Depot || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Smart Tool || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Terraforming Reactor || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Xenosilicate Template || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aborigines of the Pacifica Archipelago'' || ''206'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Haven Cluster || 207 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Missing || 208 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;New Hawaii || 209 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Northwest Territories || 212 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Prime Meridian || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Westscape || 215 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Zion Islands || 217 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Bottoms Up || 218 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creatures of Myth and Legend'' || ''220'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Illuminated Origins || 220 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;It Feeds on your Fear || 220 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Widow's Harbingers || 221 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Aborigine Graveyard || 222 |- | '''''Chapter 4: A World of Hurt''''' || '''''223''''' |- | &nbsp;&nbsp;'''The GEO''' || '''224''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Executive Council'' || ''224'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Commissioner General || 224 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Deputy Commissioner General || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;''High Commissions'' || ''226'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Human Resources || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Natural Resources || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Science and Technology || 227 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Communications || 227 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Internal Security || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State and Internal Affairs || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armed Forces || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Justice || 231 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Trade and Industry || 231 |- | &nbsp;&nbsp;&nbsp;&nbsp;''General Assembly'' || ''232'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''World Court'' || ''233'' |- | &nbsp;&nbsp;'''The Incorporated City-States''' || '''233''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Anasi Systems'' || ''233'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Atlas Materials'' || ''233'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Biogene'' || ''234'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dundalk Shipbuilding'' || ''235'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Gendiver'' || ''235'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hanover Industries'' || ''236'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hydrospan'' || ''237'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Lavender Organics'' || ''238'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''MacLeod Enforcement'' || ''238'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Nippon Industrial State'' || ''239'' |- | &nbsp;&nbsp;'''Free Zones''' || '''240''' |- | &nbsp;&nbsp;'''Earth in 2199''' || '''241''' |- | &nbsp;&nbsp;'''The Solar System''' || '''241''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Luna'' || ''241'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 241 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout, Lunar Cities || 242 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 242 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 242 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 243 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Luna and Earth || 244 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Skyhook'' || ''244'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 246 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Mars Colony'' || ''246'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 246 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 246 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 247 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 247 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 247 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Mars and Earth || 247 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 248 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Asteroid Belt and Beyond'' || ''249'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 249 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Recent Developments || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The New Belters || 251 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hole City'' || ''251'' |} [[Category:Blue Planet]] 25b8fa58795d2e7c84c9b3e6db76784b0733f4af 0 1477 1615 2007-11-26T11:07:59Z Neil 2 wikitext text/x-wiki This is a list of what organisms are found in what biomes on Poseidon. === Key === MG = Moderators' Guide, NS = Natural Selection. For threat and resource levels, 0 = None, L = Low, M = Medium, H = High, VH = Very High. For abundance, 4 = Very Common, 3 = common, 2 = uncommon, 1 = rare. {| cellpadding=2 BORDER=1 |- ! colspan=5| &nbsp; ! colspan=18 align=center| Tropical ! colspan=13 align=center| Sub-tropical ! colspan=11 align=center| Temperate ! colspan=7 align=center| Polar |- ! Organism !! Reference !! Threat level !! Resource level !! Notes !! Rainforest !! Creeper forest !! Dry tropical forest !! Montane forest !! Tropical savanna !! Coast !! Tidal pool !! Estuary !! Marsh !! Bog !! Mangrove !! Canyonland !! Shallows and shelves !! Reef !! Seamount !! Open ocean surface !! Open ocean floor !! Sargassum island !! Montane forest !! Desert !! Coast !! Tidal pool !! Estuary !! Marsh !! Mangrove !! Shallows and shelves !! Reef !! Seamount !! Open ocean surface !! Open ocean floor !! Sargassum island !! Temperate forest !! Desert !! Coast !! Tidal pool !! Estuary !! Marsh !! Tidal mud reef !! Shallows and shelves !! Seamount !! Open ocean surface !! Open ocean floor !! Tundra !! Tidal pool !! Thermal Oasis !! Shallows and shelves !! Seamount !! Open ocean surface !! Open ocean floor |- | ''Am-Bush'' || MG 163 || M || M || Westscape and surroundings || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Angel Wings'' || NS 63 || 0 || 0 || Parasitic larvae || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; |- | ''Aurora Borialgae'' || NS 64 || L || ? || Makes up Skyscraper Reefs || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Bad Mojo'' || NS 65 || M || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Basilisk'' || NS 66 || 0 || M || &nbsp; || 3 || 3 || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Big Round Thing'' || NS 67 || L || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; |- | ''Blimp'' || MG 164 || M || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Blood Hunter'' || NS 69 || VH || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Bubble Array'' || NS 70 || L || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Carniflora'' || MG 165 || H || M || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Chain Beetle'' || MG 166 || H || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Chub'' || NS 71 || 0 || H || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Cup Sucker'' || NS 72 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Digger Crab'' || MG 167 || L || H || Mainly Haven || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Dune Creeper'' || NS 73 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Echo/Fish'' || NS 74 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Eel Dragon'' || MG 167 || 0 || L || Aircraft hazard || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Fast Fungus'' || MG 168 || L || 0 || &nbsp; || 3 || 3 || &nbsp; || 3 || 3 || 3 || 3 || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || 3 || &nbsp; || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Fish'' || MG 169 || Varies || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || 3 || 3 || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || 3 || &nbsp; || 3 || 3 || 3 || 3 |- | ''Fisherman'' || MG 170 || L || M || Haven Cluster || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Ghoster'' || MG 170 || H || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Gladiator Crab'' || NS 75 || M || H || Throughout Pacifica || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Glass Coral'' || NS 76 || M || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Greater White'' || MG 171 || VH || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 |- | ''Grendel'' || NS 78 || &nbsp; || &nbsp; || New Jamaica || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hangin' Joe'' || MG 172 || H || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hard Spurt'' || MG 187 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; |- | ''Harvester Worm'' || MG 173 || L || L || Long John slag, Pebble Rocks region || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hatchlings'' || MG 173 || H || 0 || Freshwater only || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hellbender'' || NS 79 || M || L || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hexa Boar'' || MG 174 || M || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Howell's Leech'' || MG 175 || H || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Jellyroll'' || NS 80 || L || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || -3 || -3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || -3 || -3 || &nbsp; || &nbsp; || -2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Jump Jump'' || NS 81 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Keel Vine'' || NS 82 || M || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Land Lizard'' || MG 175 || H || M || &nbsp; || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Lesser White'' || NS 85 || H || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; |- | ''Leviathan'' || NS 8C || M || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; |- | ''Loggerhead'' || MG 176 || H || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Marine Iguana'' || &nbsp; || L || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Marsh Devil'' || MG 177 || H || 0 || Sierra Neuva || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Needle Bush'' || MG 177 || M || H || &nbsp; || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Needle Shell'' || MG 178 || M || M || Skyscraper reefs, Westscape || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Night Crawler'' || MG 179 || M || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Ni&ntilde;o Muerto'' || NS 86 || H || 0 || Zion, Channel, Prime Meridian || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Nooniebird'' || MG 179 || 0 || M || Westscape and surroundings || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Pharium'' || NS 87 || L || H || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Polypod'' || MG 180 || VH / 0 || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || 2 || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || 2 || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Kelp'' || MG 181 || L || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Mangrove'' || MG 182 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Potato'' || NS 88 || 0 || H || &nbsp; || 4 || 4 || 4 || 4 || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Sargassum'' || NS 89 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Scorpion'' || MG 182 || VH || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Trilobyte'' || MG 183 || L || H || New Hawaii || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Pump Weed'' || NS 91 || 0 || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Reefer Colony'' || NS 92 || 0 || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Reefworm'' || MG 184 || 0 || M || Northwest Territories || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Rubber Shrimp'' || MG 184 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; |- | ''Rumble Bee'' || MG 185 || 0 || H || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Saltwater Pseudoeel'' || NS 95 || H || L || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || 2 || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Sand Archer'' || NS 93 || M || L || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Schooler'' || MG 185 || 0 || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || 2 || 2 || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; |- | ''Sea Ghoul'' || NS 94 || 0 || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 4 || &nbsp; || 4 || 4 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 4 || 4 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 4 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 3 || &nbsp; || &nbsp; || &nbsp; |- | ''Seaweaver'' || MG 186 || ? || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Singer-in-the-Dark'' || NS 96 || H || ? || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Snow Weasel'' || NS 97 || M || ? || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; |- | ''Soft Spurt'' || MG 187 || H || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; |- | ''Squealers, Stick Monkeys'' || MG 188 || 0 || 0 || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Stone Snake'' || MG 188 || VH || 0 || Zion islands || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''String Worm'' || NS 98 || H || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Sunburst'' || MG 189 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; |- | ''Sweet Noodles'' || NS 99 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Thornrow'' || NS 100 || L || M || Westscape and surroundings || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Trident Fish'' || MG 190 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; |- | ''Walkabout'' || MG 191 || H || ? || Prime Meridian || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Water Dart'' || MG 192 || H || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || ? || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Water Hemp'' || MG 192 || 0 || H || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Water Rat'' || MG 193 || 0 || 0 || Zion islands || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Weedeater'' || NS 101 || 0 || H || Kelp forests || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Wraparound'' || NS 103 || L || 0 || Fish parasite || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Xenosilicabenthoid'' || NS 104 || 0 || ? || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Zipper'' || NS 104 || L || H || Zion islands, NW Territories || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |} === Other unspecified creatures === {| cellpadding=3 border=1 |- | ''Wash'' || NS 17 |- | ''Cuddlesloth'' || NS 18 |- | ''Alex's Ribbonsquid'' || NS 21 |- | ''Storm Children'' || NS 23 |- | ''Articulated Rook'' || NS 24 |- | ''Creeping Towel'' || NS 24 |} [[Category:Blue Planet]] 4a7a2c7168291cf6af3f50ed99a5774cecad14ad 0 1478 1618 2007-11-26T14:40:47Z Neil 2 wikitext text/x-wiki These are some useful summary charts for Synergy game system, as used the [[:Category:Blue Planet|Blue Planet v2]] role-playing game. {| |- valign="top" | {| ! Task difficulty |- | {| | Foolproof || +5 |- | Routine || +3 |- | Easy || +1 |- | Average || - |- | Complicated || -1 |- | Challenging || -3 |- | Formidable || -5 |} |- ! Range |- | {| | Point-blank || +2 |- | Short || +0 |- | Medium || -2 |- | Long || -4 |} |- ! Close Combat |- | {| | Per additional attacker || -1 |- | Active defence || +2 |} |- ! Pace |- | {| ! !! Target !! Attacker |- | Walk/Crawl || -1 || -2 |- | Scramble || -2 || -4 |- | Sprint || -4 || X |} |- ! Called Shot |- | -''n'' to hit, &plusmn;''n'' to damage |} | {| ! Other modifiers |- | Recoil || -2 for each additional fire action |- | Aiming || +1 per action |- | Poor visibility || -2 |- | Partial cover || -2 |- | Full cover || -4 |- | Blind fire || -5 |- | Underwater || -3 |- | Unstable footing || -1 |- | Firing off-hand || -2 |- | Firing pistol one-handed || -1 |- | Firing rifle one-handed || -3 |- | Under heavy fire || -2 |- | Target prone || -2 |- | Height advantage || +1 |- | Firearm supported || +1 |- | Flank attack || +1 |- | Rear attack || +2 |- | Additional simultaneous action || -2 to all actions |} |} {| ! Armour ratings |- | Wooden wall || 2 |- | Biocrete wall || 6 |- | Tree || 5 |- | Brush || 1 |} == Wounds == Attacker roll 3 dice with target of [Damage - (Toughness + Armour)]. Count successes. {| ! Successes !! Wound level !! Effects |- | 0 || Negligible || No effect |- | 1 || Minor || -1 penalty |- | 2 || Serious || -2 penalty.<br>Will save at -2 or impaired until first aid (AV &le; -5 &rarr; unconscious) |- | 3 || Critical || -3 penalty.<br>Fitness save at -3 or mortal wound (AV &le; -5 &rarr; instant death)<br>Will save at -3 or incapacitated (AV &le; -5 &rarr; unconscious) [[Category:Blue Planet]] |} [[Category:Blue Planet]] 9373e22a9b7943ac3569d661d9198afdc96cf69d 0 1480 1620 2007-11-26T14:55:23Z Neil 2 wikitext text/x-wiki Gareth Hanrahan's scenario ''[http://www.gameeire.com/Scenarios/bplanet.html Treacherous Waters]'' is a wonderful blood opera, but the files don't include the statistics for the pregenerated characters. Here are my versions of them. Note that: # These are slightly underpowered Exceptional characters: they have the skill packages of Exceptional characters, but the attributes of Everyday characters; # Superscripts on skill ratings show the number of dice to roll for that skill (I can't keep track of aptitudes when I'm running the game); # Only exceptional equipment is listed here: common-or-garden equipment, including guns and armour, is available on the station. == Cmdr Alan Baum == * ''Packages'': Origin: GEO; Backgrounds: GEO, University; Professional: Expert Espionage (Analyst), Specialist Military. * ''Primary Attributes'': Build -1, Fitness 0, Agility 0, Dexterity 0, Awareness +1 (Chemical 0, Hearing +1, Intuition +2, Touch 0, Vision +2), Intellect +2, Presence -1, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 5; Aquatics 5, Throwing 4; Armed Combat 2, Unarmed Combat 4; Leadership 2&sup2;, Logistics 2&sup2;, Strategy 4&sup2;, Tactics 2&sup2;; Persuasion 5&sup3;, Writing 4&sup3;; Culture (Earth) 5&sup2;, Culture (GEO) 9&sup2;, Culture (Incorporate) 6&sup2;, Culture (Military) 3&sup2;; Handguns 4, Longarms 3; History 2, Politics 6; First Aid 2; Heavy Weapons 2; Hiding 2&sup2;, Shadowing 2&sup2;, Sneaking 2&sup2;; Bribery 2&sup2;, Disguise 2&sup2;, Fast Talk 4&sup2;, Forgery 1&sup2;, Lockpicking 2&sup2;; Computers 6&sup3;, Demolitions 1&sup3;, Electronics 6&sup3;, Remote Operations 4&sup3;. * ''Mods and equipment'': Diver genmod, neural jack, heavy vest, combat helmet. == Dr Faith Tourmanlene == * ''Packages'': Origin: Earth Orbit; Backgrounds: GEO, University; Professional: Expert Science, Novice Admin, Novice Survival. * ''Primary Attributes'': Build -2, Fitness -1, Agility -2, Dexterity 0, Awareness 0 (Chemical 0, Hearing -1, Intuition -1, Touch +1, Vision +1), Intellect +3, Presence +1, Will +1. * ''Secondary Attributes'': Endurance 0, Reflexes -1, Strength -1, Toughness -1. * ''Skills'': Bureaucracy 6&sup2;, Economics 1&sup2;, Law 1&sup2;, Management 1&sup2;; Aquatics 1, Freefall 1, Throwing 2; Leadership 1, Logistics 1; Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 4&sup3;, Writing 3&sup3;; Culture (Earth) 5&sup2;, Culture (GEO) 4&sup2;, Culture (Incorporate) 3&sup2;, Culture (Native) 2&sup2;; Longarms 2; History 2, Politics 3; Biochemistry 7&sup3;, Botany 7&sup3;, Genetics 5&sup3;, Zoology 6&sup3;; First Aid 1; Astronomy 3&sup2;, Chemistry 3&sup2;, Geology 3&sup2;, Meterology 4&sup2;, Physics 3&sup2;; Hiding 1, Sneaking 2; Fast Talk 2; Fishing 2, Foraging 2, Mountaineering 2, Navigation 1, Orienteering 2; Computers 7&sup2;, Damage Control 1&sup2;, Electronics 1&sup2;, Mechanics 1&sup2;, Remote Operations 3&sup2;; Piloting 2. * ''Mods and equipment'': Neural jack, anti-poison, salt tolerance, multiglands. 1f8f0f4d44c7dd444cfed66dd0461af8e52575e1 1621 1620 2007-11-26T14:55:42Z Neil 2 wikitext text/x-wiki Gareth Hanrahan's scenario ''[http://www.gameeire.com/Scenarios/bplanet.html Treacherous Waters]'' is a wonderful blood opera, but the files don't include the statistics for the pregenerated characters. Here are my versions of them. Note that: # These are slightly underpowered Exceptional characters: they have the skill packages of Exceptional characters, but the attributes of Everyday characters; # Superscripts on skill ratings show the number of dice to roll for that skill (I can't keep track of aptitudes when I'm running the game); # Only exceptional equipment is listed here: common-or-garden equipment, including guns and armour, is available on the station. == Cmdr Alan Baum == * ''Packages'': Origin: GEO; Backgrounds: GEO, University; Professional: Expert Espionage (Analyst), Specialist Military. * ''Primary Attributes'': Build -1, Fitness 0, Agility 0, Dexterity 0, Awareness +1 (Chemical 0, Hearing +1, Intuition +2, Touch 0, Vision +2), Intellect +2, Presence -1, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 5; Aquatics 5, Throwing 4; Armed Combat 2, Unarmed Combat 4; Leadership 2&sup2;, Logistics 2&sup2;, Strategy 4&sup2;, Tactics 2&sup2;; Persuasion 5&sup3;, Writing 4&sup3;; Culture (Earth) 5&sup2;, Culture (GEO) 9&sup2;, Culture (Incorporate) 6&sup2;, Culture (Military) 3&sup2;; Handguns 4, Longarms 3; History 2, Politics 6; First Aid 2; Heavy Weapons 2; Hiding 2&sup2;, Shadowing 2&sup2;, Sneaking 2&sup2;; Bribery 2&sup2;, Disguise 2&sup2;, Fast Talk 4&sup2;, Forgery 1&sup2;, Lockpicking 2&sup2;; Computers 6&sup3;, Demolitions 1&sup3;, Electronics 6&sup3;, Remote Operations 4&sup3;. * ''Mods and equipment'': Diver genmod, neural jack, heavy vest, combat helmet. == Dr Faith Tourmanlene == * ''Packages'': Origin: Earth Orbit; Backgrounds: GEO, University; Professional: Expert Science, Novice Admin, Novice Survival. * ''Primary Attributes'': Build -2, Fitness -1, Agility -2, Dexterity 0, Awareness 0 (Chemical 0, Hearing -1, Intuition -1, Touch +1, Vision +1), Intellect +3, Presence +1, Will +1. * ''Secondary Attributes'': Endurance 0, Reflexes -1, Strength -1, Toughness -1. * ''Skills'': Bureaucracy 6&sup2;, Economics 1&sup2;, Law 1&sup2;, Management 1&sup2;; Aquatics 1, Freefall 1, Throwing 2; Leadership 1, Logistics 1; Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 4&sup3;, Writing 3&sup3;; Culture (Earth) 5&sup2;, Culture (GEO) 4&sup2;, Culture (Incorporate) 3&sup2;, Culture (Native) 2&sup2;; Longarms 2; History 2, Politics 3; Biochemistry 7&sup3;, Botany 7&sup3;, Genetics 5&sup3;, Zoology 6&sup3;; First Aid 1; Astronomy 3&sup2;, Chemistry 3&sup2;, Geology 3&sup2;, Meterology 4&sup2;, Physics 3&sup2;; Hiding 1, Sneaking 2; Fast Talk 2; Fishing 2, Foraging 2, Mountaineering 2, Navigation 1, Orienteering 2; Computers 7&sup2;, Damage Control 1&sup2;, Electronics 1&sup2;, Mechanics 1&sup2;, Remote Operations 3&sup2;; Piloting 2. * ''Mods and equipment'': Neural jack, anti-poison, salt tolerance, multiglands. [[Category:Blue Planet]] 3be9e62d8bbdb3122584d20521e3556b3dbe936e 1622 1621 2007-11-26T16:23:13Z Neil 2 wikitext text/x-wiki Gareth Hanrahan's scenario ''[http://www.gameeire.com/Scenarios/bplanet.html Treacherous Waters]'' is a wonderful blood opera, but the files don't include the statistics for the pregenerated characters. Here are my versions of them. Note that: # These are slightly underpowered Exceptional characters: they have the skill packages of Exceptional characters, but the attributes of Everyday characters; # Superscripts on skill ratings show the number of dice to roll for that skill (I can't keep track of aptitudes when I'm running the game); # Only exceptional equipment is listed here: common-or-garden equipment, including guns and armour, is available on the station. == Cmdr Alan Baum == * ''Packages'': Origin: GEO; Backgrounds: GEO, University; Professional: Expert Espionage (Analyst), Specialist Military. * ''Primary Attributes'': Build -1, Fitness 0, Agility 0, Dexterity 0, Awareness +1 (Chemical 0, Hearing +1, Intuition +2, Touch 0, Vision +2), Intellect +2, Presence -1, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 5; Aquatics 5, Throwing 4; Armed Combat 2, Unarmed Combat 4; Leadership 2&sup2;, Logistics 2&sup2;, Strategy 4&sup2;, Tactics 2&sup2;; Persuasion 5&sup3;, Writing 4&sup3;; Culture (Earth) 5&sup2;, Culture (GEO) 9&sup2;, Culture (Incorporate) 6&sup2;, Culture (Military) 3&sup2;; Handguns 4, Longarms 3; History 2, Politics 6; First Aid 2; Heavy Weapons 2; Hiding 2&sup2;, Shadowing 2&sup2;, Sneaking 2&sup2;; Bribery 2&sup2;, Disguise 2&sup2;, Fast Talk 4&sup2;, Forgery 1&sup2;, Lockpicking 2&sup2;; Computers 6&sup3;, Demolitions 1&sup3;, Electronics 6&sup3;, Remote Operations 4&sup3;. * ''Mods and equipment'': Diver genmod, neural jack, heavy vest, combat helmet. == Dr Faith Tourmanlene == * ''Packages'': Origin: Earth Orbit; Backgrounds: GEO, University; Professional: Expert Science, Novice Admin, Novice Survival. * ''Primary Attributes'': Build -2, Fitness -1, Agility -2, Dexterity 0, Awareness 0 (Chemical 0, Hearing -1, Intuition -1, Touch +1, Vision +1), Intellect +3, Presence +1, Will +1. * ''Secondary Attributes'': Endurance 0, Reflexes -1, Strength -1, Toughness -1. * ''Skills'': Bureaucracy 6&sup2;, Economics 1&sup2;, Law 1&sup2;, Management 1&sup2;; Aquatics 1, Freefall 1, Throwing 2; Leadership 1, Logistics 1; Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 4&sup3;, Writing 3&sup3;; Culture (Earth) 5&sup2;, Culture (GEO) 4&sup2;, Culture (Incorporate) 3&sup2;, Culture (Native) 2&sup2;; Longarms 2; History 2, Politics 3; Biochemistry 7&sup3;, Botany 7&sup3;, Genetics 5&sup3;, Zoology 6&sup3;; First Aid 1; Astronomy 3&sup2;, Chemistry 3&sup2;, Geology 3&sup2;, Meterology 4&sup2;, Physics 3&sup2;; Hiding 1, Sneaking 2; Fast Talk 2; Fishing 2, Foraging 2, Mountaineering 2, Navigation 1, Orienteering 2; Computers 7&sup2;, Damage Control 1&sup2;, Electronics 1&sup2;, Mechanics 1&sup2;, Remote Operations 3&sup2;; Piloting 2. * ''Mods and equipment'': Neural jack, anti-poison, salt tolerance, multiglands. == Dr Tsung-Li == * ''Packages'': Origin: Freezone&emdash;Wasteland; Backgrounds: Incorporate, University; Professional: Expert Science, Novice Espoinage, Novice Technical. * ''Primary Attributes'': Build -2, Fitness 0, Agility -1, Dexterity 0, Awareness 0 (Chemical -1, Hearing 0, Intuition +1, Touch +1, Vision -1), Intellect +2, Presence -1, Will +1. * ''Secondary Attributes'': Endurance 0, Reflexes 0, Strength -1, Toughness 0. * ''Skills'': Bureaucracy 5, Economics 1, Law 1; Unarmed Combat 1; Persuasion 4&sup2;, Writing 4&sup2;; Culture (Earth) 5&sup2;, Culture (Incorporate) 3&sup2;, Culture (Military) 2&sup2;; Handguns 1, Longarms 1; History 1, Politics 2; Biochemistry 3&sup2;, Botany 3&sup2;, Genetics 3&sup2;, Zoology 3&sup2;; First Aid 1; Astronomy 5&sup3;, Chemistry 5&sup3;, Geology 5&sup3;, Meterology 7&sup3;, Physics 7&sup3;; Hiding 1, Shadowing 1, Sneaking 1; Bribery 1, Disguise 1, Fast Talk 4, Lockpicking 3; Foraging 1, Navigation 3, Orienteering 1, Tracking 1; Computers 8&sup3;, Damage Contol 2&sup3;, Electronics 3&sup3;, Mechanics 3&sup3;, Remote Operations 6&sup3;; Driving 2, Piloting 2. * ''Mods and equipment'': Uplink neural jack, implanted computer, audio/visual bug suite. == Lt Feyd Al-Umarj == * ''Packages'': Origin: Mars; Backgrounds: GEO, Colonial; Professional: Expert Military (Marine), Specialist Survival. * ''Primary Attributes'': Build +1, Fitness +2, Agility +2, Dexterity +1, Awareness 0 (Chemical 0, Hearing +1, Intuition -1, Touch 0, Vision 0 [+2 in low light]), Intellect -2, Presence 0, Will -1. * ''Secondary Attributes'': Endurance +1, Reflexes +2, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 2; Aquatics 7&sup2;, Parachuting 2&sup2;, Throwing 4&sup2;; Armed Combat 6&sup3;, Unarmed Combat 5&sup3;; Leadership 3, Strategy 3, Tactics 3; Culture (Cetacean) 1, Culture (Colonial) 3, Culture (GEO) 4, Culture (Incorporate) 1, Culture (Military) 5, Culture (Native) 1, Culture (Spacer) 3; Handguns 6&sup3;, Longarms 7&sup3;; History 1, Politics 1; First Aid 5; Heavy Weapons 5; Meterology 2; Hiding 3&sup2;, Sneaking 5&sup2;; Fishing 3, Foraging 3, Mountaineering 3, Navigation 3, Orienteering 4, Tracking 3; Computers 2, Damage Control 1, Demolitions 1, Electronics 2, Mechanics 1, Remote Operations 3; Driving 3&sup2;, Piloting 2&sup2;, Sailing 3&sup2;. * ''Mods and equipment'': Diver genmod, accelerated neurons, infrared sensors, subdermal plates, adrenal shunt, salt tolerance, heavy vest, combat helmet. == Dr Vlad Olyah == * ''Packages'': Origin: GEO; Backgrounds: GEO, University; Professional: Specialist Science, Novice Admin, Novice Survival, Novice Arts & Entertainment. * ''Primary Attributes'': Build -1, Fitness +1, Agility 0, Dexterity 0, Awareness 0 (Chemical -1, Hearing +1, Intuition 0, Touch +1, Vision -1), Intellect +1, Presence +1, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 52, Economics 12, Law 12; Aquatics 4; Leadership 1, Logistics 1; Interspec 3&sup3;, Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 3&sup3;, Writing 5&sup3;; Culture (Earth) 7&sup2;, Culture (GEO) 7&sup2;, Culture (Incorporate) 2&sup2;, Culture (Native) 4&sup2;; Acting 5, Dance 3, Music 6; Longarms 1; History 2, Politics 4; Biochemistry 4&sup3;, Botany 5&sup3;, Genetics 4&sup3;, Zoology 5&sup3;; First Aid 1; Astronomy 2&sup2;, Chemistry 2&sup2;, Geology 2&sup2;, Meterology 3&sup2;, Physics 2&sup2;; Hiding 1, Sneaking 2; Fast Talk 2; Fishing 2, Foraging 2, Mountaineering 2, Navigation 1, Orienteering 2, Tracking 2; Computers 7&sup2;, Mechanics 1&sup2;, Remote Operations 2&sup2;. * ''Mods and equipment'': Diver genmod, perfect pitch, multiglands. == Martha Grosse == * ''Packages'': Origin: Urban; Backgrounds: Incorporate, Independent; Professional: Expert Espionage (Operative), Novice Entertainment, Novice Survival. * ''Primary Attributes'': Build -1, Fitness +2, Agility +3, Dexterity +2, Awareness 0 (Chemical -1, Hearing +1, Intuition 0, Touch -1, Vision +1), Intellect -1, Presence -2, Will 0. * ''Secondary Attributes'': Endurance +2, Reflexes +2, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 3, Economics 3, Management 1; Aquatics 5; Unarmed Combat 3&sup2;; Negotiation 6&sup2;, Persuasion 6&sup2;; Culture (Earth) 6, Culture (GEO) 2, Culture (Incorporate) 6, Culture (Street) 2; Acting 5, Dance 3, Photography 3; Handguns 3, Longarms 3; Politics 2; Botany 1, Zoology 1; First Aid 2; Meterology 1; Hiding 5&sup3;, Shadowing 3&sup3;, Sneaking 3&sup3;; Bribery 3&sup3;, Disguise 2&sup3;, Fast Talk 5&sup3;, Lockpicking 3&sup3;; Fishing 2, Foraging 2, Mountaineering 2, Navigation 1, Orienteering 2, Tracking 2; Computers 7&sup2;, Demolitions 3&sup2;, Electronics 3&sup2;, Remote Operations 3&sup2;; Driving 4&sup2;, Piloting 3&sup2;. * ''Mods and equipment'': Diver genmod, implanted sensory recorder, accelerated neurons, neural jack. == Native Ecoterrorists == * Reflexes 0, Agility 0, Dexterity 0, Toughness +1, Fitness +1, Will +1, Armour 4. * SMG 3&sup2;, DR 6/9. * Assault Rifle 5&sup2;, DR 11. * Grenade 5&sup2;, DR 8. * Knife 3&sup3;, DR 2. [[Category:Blue Planet]] 2022e1249b120e23a6945961fa04d4d51c9b984a 0 1482 1624 2007-11-26T21:42:23Z Neil 2 [[Blue Planet house rules]] moved to [[Synergy house rules]] wikitext text/x-wiki #REDIRECT [[Synergy house rules]] c652d6002294f9a6f2dee87a4be1ce49878dc535 0 1483 1626 2007-11-26T22:02:29Z Neil 2 wikitext text/x-wiki Gareth Hanrahan's scenario ''[http://www.gameeire.com/Scenario/bplanet.html Things to do in Dyfedd When You're Genlifted]'' is a great crime caper, but the files don't include the statistics for the pregenerated characters. Here are my versions of them. Note that: # These are slightly underpowered Exceptional characters: they have the skill packages of Exceptional characters, but the attributes of Everyday characters; # Superscripts on skill ratings show the number of dice to roll for that skill (I can't keep track of aptitudes when I'm running the game). == Serge Klazimakoff == * ''Packages'': Origin: Freezone&emdash;Wasteland; Backgrounds: Colonial, Street; Professional: Expert Crime (Gangster), Novice Technical, Novice Commerce. * ''Primary Attributes'': Build +2, Fitness +2, Agility -1, Dexterity 0, Awareness -1 (Chemical -2, Hearing -1, Intuition -1, Touch -1, Vision 0), Intellect +1, Presence -1, Will 0. * ''Secondary Attributes'': Endurance +2, Reflexes -1, Strength +2, Toughness +1. * ''Skills'': Bureaucracy 4, Law 6; Aquatics 4; Armed Combat 4&sup2;, Unarmed Combat 5&sup2;; Leadership 2; Negotiation 5&sup2;, Persuasion 5&sup2;; Culture (Cetacean) 1, Culture (Colonial) 3, Culture (Earth) 3, Culture (GEO) 1, Culture (Incorporate) 2, Culture (Native) 1, Culture (Street) 6; Handguns 5, Longarms 5; First Aid 1, Forensic Medicine 1, Psychology 1; Physics 1; Hiding 4&sup2;, Shadowing 4&sup2;, Sneaking 4&sup2;; Bribery 5&sup3;, Fast Talk 6&sup3;, Lockpicking 4&sup3;, Sleight of Hand 1&sup3;; Foraging 1, Orienteering 1, Tracking 1; Computers 2&sup2;, Damage Control 2&sup2;, Electronics 2&sup2;, Mechanics 2&sup2;, Remote Operations 3&sup2;; Driving 4&sup3;, Piloting 4&sup3;. * ''Modifications'': Improved blood oxygenation, immunological symbiote, salt tolerance, pain inhibitor. * ''Equipment'': Bodycomp, utility jumpcraft, large calibre handgun, sub-machine gun, light armour vest, reinforced armour vest. == Larry Scjelli == * ''Species'': Transhuman. * ''Packages'': Origin: Incorporate; Backgrounds: Incorporate, Wealthy; Professional: Specialist Administration, Specialist Crime, Novice Diplomacy. * ''Primary Attributes'': Build 0, Fitness +1, Agility -1, Dexterity -1, Awareness +1 (Chemical +1, Hearing 0, Intuition +2, Touch +1, Vision +1), Intellect +2, Presence +4, Will +2. * ''Secondary Attributes'': Endurance +2, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 6&sup3;, Economics 5&sup3;, Law 4&sup3;, Management 3&sup3;; Armed Combat 2, Unarmed Combat 2; Leadership 4&sup2;, Logistics 2&sup2;, Strategy 3&sup2;; Language (English, Native), Language (Arabic) 5&sup3;, Language (German) 3&sup3;, Negotiation 6&sup3;, Oration 3&sup3;, Persuasion 6&sup3;, Writing 3&sup3;; Culture (Colonial) 3&sup2;, Culture (Earth) 2&sup2;, Culture (GEO) 3&sup2;, Culture (Incorporate) 8&sup2;, Culture (Native) 3&sup2;, Culture (Street) 3&sup2;; Painting 1; Handguns 2; History 22, Politics 42; Psychology 1; Hiding 3, Shadowing 3, Sneaking 3; Bribery 3&sup2;, Fast Talk 6&sup2;, Lockpicking 2&sup2;, Sleight of Hand 1&sup2;; Computers 3; Driving 1, Piloting 3. * ''Modifications'': Bodysculpting, immunological symbiote, multiglands, neural jack. * ''Equipment'': Minisub, concealable handgun. == Li Fan Tsu == * ''Packages'': Origin: Urban; Backgrounds: GEO, Independent; Professional: Expert Law Enforcement (Undercover), Novice Crime, Novice Athletics. * ''Primary Attributes'': Build 0, Fitness +3, Agility +2, Dexterity +2, Awareness +2 (Chemical +1, Hearing +5, Intuition +2, Touch +1, Vision +3), Intellect 0, Presence 0, Will -1. * ''Secondary Attributes'': Endurance 3, Reflexes 3, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 5&sup2;, Economics 2&sup2;, Law 5&sup2;; Aquatics 3&sup2;, Sports 3&sup2;, Throwing 3&sup2;; Armed Combat 3&sup2;, Unarmed Combat 5&sup2;; Language: Interspec 1, Negotiation 3, Persuasion 6; Culture (Colonial) 2, Culture (Earth) 5, Culture (GEO) 4, Culture (Incorporate) 2, Culture (Native) 3, Culture (Street) 6; Acting 3, Photography 3; Handguns 5&sup3;, Longarms 2&sup3;; History 1, Politics 1; First Aid 5, Forensic Medicine 3, General Medicine 1, Pharmacology 1; Hiding 3&sup3;, Shadowing 5&sup3;, Sneaking 3&sup3;; Bribery 1&sup2;, Fast Talk 4&sup2;, Lockpicking 3&sup2;; Computers 3; Driving 6. * ''Modifications'': Amplified hearing, improved blood oxygenation, immunological symbiote, salt tolerance, accelerated neurons, neural jack, programmed reflexes (autoload, quick draw, defender). * ''Equipment'': Bodycomp + recorders, medical kit, diving mask + artificial gill, powerboat, diamond knife, concealable handgun, large calibre handgun, light armour vest. == Abdullah Al-Maluk == * ''Packages'': Origin: Urban; Backgrounds: Incorporate, University; Professional: Expert Espionage (Operative), Novice Crime, Novice Diplomacy. * ''Primary Attributes'': Build 0, Fitness 0, Agility 0, Dexterity +1, Awareness 0 (Chemical 0, Hearing 0, Intuition +1, Touch -1, Vision 0), Intellect +1, Presence +1, Will 0. * ''Secondary Attributes'': Endurance 0, Reflexes +1, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 7, Economics 1, Law 1, Management 1; Armed Combat 4&sup2;, Unarmed Combat 4&sup2;; Strategy 1; Language (Arabic, Native), English 8&sup3;, Spanish 3&sup3;, Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 6&sup3;, Writing 3&sup3;; Culture (Colonial) 4&sup3;, Culture (Earth) 7&sup3;, Culture (Incorporate) 5&sup3;, Culture (Street) 5&sup3;; Handguns 4, Longarms 2; History 2, Politics 3, Theology 1; Psychology 1; Hiding 5, Shadowing 5, Sneaking 5; Bribery 1&sup2;, Disguise 2&sup2;, Fast Talk 6&sup2;, Lockpicking 4&sup2;; Computers 6&sup2;, Demolitions 3&sup2;, Electronics 3&sup2;, Remote Operations 3&sup2;; Driving 2. * ''Modifications'': Implanted sensory recorder, immunological symbiote, accelerated neurons, uplink neural jack. * ''Equipment'': Superior bodycomp, pain inhibitor drug &times;2, reflex serum &times;2, ultrasonic hypo, light armour vest, diamond knife, concealable handgun, small calibre handgun. == Genevive Sian == * ''Species'': Native Aquaform (Diver). * ''Packages'': Origin: Native; Backgrounds: Native, Colonial; Professional: Specialist Survival, Specialist Military, Novice Crime. * ''Primary Attributes'': Build -2, Fitness +1, Agility +1, Dexterity 0, Awareness +1 (Chemical +1, Hearing +2, Intuition +1, Touch +1, Vision 0), Intellect -1, Presence 0, Will +1. * ''Secondary Attributes'': Endurance +2, Reflexes +1, Strength 0, Toughness 0. * ''Skills'': Aquaculture 2; Aquatics 10&sup2;, Throwing 4&sup2;; Armed Combat 4&sup2;, Unarmed Combat 4&sup2;; Leadership 2, Strategy 2, Tactics 2; Interspec 4, Negotiation 1, Persuasion 1; Culture (Cetacean) 3, Culture (Colonial) 6, Culture (GEO) 1, Culture (Military) 1, Culture (Native) 6, Culture (Street) 1; Handguns 3&sup2;, Longarms 5&sup2;; Botany 2, Zoology 2; First Aid 4; Heavy Weapons 2; Meterology 2; Hiding 4&sup3;, Shadowing 2&sup3;, Sneaking 4&sup3;; Bribery 3, Fast Talk 1, Lockpicking 1; Fishing 5&sup3;, Foraging 3&sup3;, Mountaineering 3&sup3;, Navigation 4&sup3;, Orienteering 4&sup3;, Tracking 3&sup3;; Demolitions 1, Remote Operations 2; Driving 1&sup2;, Piloting 1&sup2;, Sailing 5&sup2;. * ''Modifications'': None. * ''Equipment'': Portable navigation suite, native catamaran, diamond knife, small calibre handgun, hunting rifle + optical scope. == Marlon Clark == * ''Packages'': Origin: Mars Colony; Backgrounds: Criminal, Street; Professional: Expert Crime (Thug), Novice Law Enforcement, Novice Espionage. * ''Primary Attributes'': Build 0, Fitness +2, Agility +2, Dexterity +2, Awareness -1 (Chemical -1, Hearing -1, Intuition -2, Touch -1, Vision 0), Intellect 0, Presence -2, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes +1, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 1, Law 6; Armed Combat 4&sup2;, Unarmed Combat 6&sup2;; Strategy 2&sup2;, Tactics 3&sup2;; Negotiation 4&sup2;, Persuasion 6&sup2;; Culture (Colonial) 2, Culture (Spacer) 3, Culture (Street) 6; Handguns 6&sup2;, Longarms 5&sup2;; Politics 1; Forensic Medicine 1; Hiding 6&sup3;, Shadowing 6&sup3;, Sneaking 6&sup3;; Bribery 6&sup3;, Disguise 2&sup3;, Fast Talk 6&sup3;, Lockpicking 5&sup3; Sleight of Hand 2&sup3;; Computers 2, Damage Control 1, Electronics 2, Mechanics 1, Remote Operations 2; Driving 3, Piloting 1. * ''Modifications'': Anti-poison, accelerated neurons, neural jack, implanted computer. * ''Equipment'': Concealable handgun, large calibre handgun, stun baton. == Gorchoff Thug == * Reflexes +1, Agility +1, Dexterity 0, Awareness 0, Toughness 0, Fitness +1, Will +1, Armour 2. * Handgun 5&sup2;, DR 6. == Dyffedd Security == * Reflexes +1, Agility +1, Dexterity 0, Awareness 0, Toughness 0, Fitness +1, Will +1, Armour 3. * SMG 6&sup2;, DR 6/9. * Stun Gun 6&sup2;, DR Special. == Free Poseidon Ecoterrorists == * Reflexes 0, Agility +2, Dexterity 0, Toughness +1, Fitness +2, Will 0, Armour 2. * Hunting Rifle 3, DR 9. * Knife 3&sup3;, DR 2. == Grimm == * Reflexes +3, Agility +2, Dexterity 0, Toughness +2, Fitness +2, Will +2, Armour 3. * Custom Handgun 8&sup3;, DR 7. == Vermier == * Reflexes +2, Agility +1 Dexterity +1, Toughness +1, Fitness 0, Will +1, Armour 0. * Shotgun 5&sup2;, DR 7/10. [[Category:Blue Planet]] e11b6cee3c7107c835bcbb3fc25a294dc6e28295 0 1484 1627 2007-11-26T22:03:17Z Neil 2 Redirecting to [[Category:Blue Planet]] wikitext text/x-wiki #REDIRECT [[:Category:Blue Planet]] 491fd9e7c2e1383f26c07c2d9d3af306100b12eb 0 1485 1628 2007-11-26T22:04:47Z Neil 2 Redirecting to [[Category:RPGs]] wikitext text/x-wiki #REDIRECT [[:Category:RPGs]] dece4bf4cf7e31f9a09c9562532edbd395001541 0 1486 1629 2007-11-26T22:05:28Z Neil 2 Redirecting to [[Category:RPGs]] wikitext text/x-wiki #REDIRECT [[:Category:RPGs]] dece4bf4cf7e31f9a09c9562532edbd395001541 0 1487 1630 2007-11-26T22:06:09Z Neil 2 Redirecting to [[Category:RPGs]] wikitext text/x-wiki #REDIRECT [[:Category:RPGs]] dece4bf4cf7e31f9a09c9562532edbd395001541 0 1488 1631 2007-11-26T22:24:24Z Neil 2 wikitext text/x-wiki A couple of Praxian-themed songs, inspired by the [[Category:Yellowtail|Yellowtail campaign]]. For more Gloranthan filk, see [http://www.etyries.com/songbook/ Nick Brooke's song page] and [http://www.jane-williams.me.uk/glorantha/songs.cfm Jane Williams's song page]. == We Will Raid You == :to the tune of Queen's ''We Will Rock You'' Pentans you're a horde make a big noise<br> Ridin' in the snow<br> Gonna be big trouble some day<br> You got hair on yo' face<br> You big disgrace<br> Ridin' your horses all over the place :We will, we will raid you<br> :We will, we will raid you Pavis you're a hard tent, rock tent<br> Pitched by the river<br> Gonna take over the valley some day<br> You got dust on yo' face<br> You big disgrace<br> Hiding from the trolls eatin' up your place :We will, we will raid you<br> :We will, we will raid you Quivin you're a high place, tall place<br> Hero says he's come,<br> Gonna make you some peace some day<br> Yoy got woad on yo' face<br> You big disgrace<br> Fightin' your feuds all over the place :We will, we will raid you<br> :We will, we will raid you<br> :We will, we will raid you<br> :We will, we will raid you<br> == We're Waha's Champions == :to the tune of Queen's ''We Are the Champions'' I've paid my dues, time after time<br> I've walked in exile, but committed no crime<br> And bad mistakes, I've made a few<br> I've had my share of sand kicked in my face, but I've come through :We're Waha's champions, my friends<br> :And we'll keep on raiding till the end<br> :We're Waha's champions, we're Waha's champions<br> :No time for losers<br> :'Cause we're Waha's champions, of the Plaines I've taken my cheers, and my tribute gifts,<br> You brought me scalps and beasts and everything that goes with it<br> I thank you all<br> But it's been no padded saddle, no Waha trail,<br> I consider it a challenge before the whole Greatland race,<br> and I ain't gonna fail :We're Waha's champions, my friends<br> :And we'll keep on raiding till the end<br> :We're Waha's champions, we're Waha's champions<br> :No time for losers<br> :'Cause we're Waha's champions, of the Plaines :We're Waha's champions, my friends<br> :And we'll keep on raiding till the end<br> :We're Waha's champions, we're Waha's champions<br> :No time for losers<br> :'Cause we're Waha's champions [[Category:Yellowtail]] 129c6d0227457e99b48c8549a176f18b28840346 0 1489 1632 2007-11-26T22:28:50Z Neil 2 wikitext text/x-wiki ''This article is expanded from my original posting to the [http://glorantha.temppeli.org/digest/ Glorantha Digest] volume 3 number 146. It contains revisions prompted by feedback from that posting.'' Elves are an enigmatic species in Glorantha. They are often met in legends and games, but we know little about their motivations and thoughts. We know that they live in forests, worship Grower, and often behave in an incomprehensible manner (as far as humans are concerned). I think that these facts are all tied together, and to understand how elves think and act, we need to understand how they perceive the world. But before we get to that, let's back up a bit. The major deity in elves' lives is Aldrya, the Goddess of Forests. That's important. She isn't the Goddess of Trees; She isn't even the Goddess of 'Lots of Trees Growing Near Each Other'. She is the Goddess of ''Forests'', the mother of the whole eco-system that is the Forest. Every tree, every bush, every earthworm, every dung beetle, every woodpecker, everything that goes towards keeping the ''Forest'' a vibrant, living organism, comes under her purview. And elves are Her way of reacting quickly to the needs of the Forest. Which brings us to elves and Elfsense. In ''Elder Secrets'', Elfsense is described as allowing the elf to discern what is wrong with a plant they touch. That sounds rather lame to me. I think that description reflects what Gloranthan humans understand about Elfsense, rather that what it really is. I think that Elfsense is the elves' primary sense, the way they gain most of their information about the world. Humans have vision, trolls have hearing, dwarves have touch, and in an identical way, elves have empathy. We know that in the real world, plants communicate with each other; that if one tree is attacked by pests, its neighbours take pre-emptive steps to protect themselves. That is what Elfsense is: the elves' perception of the constant communication between plants as they experience and react to the world around them. This means that Elfsense has two 'modes', in the same way that human vision has two 'modes'. Normally, as elves go about their everyday business, they perceive the background 'hum' of the Forest living, as plants grow, feed, die, rot. This "Song of the Forest" is constantly 'heard' by everything in the Forest. As the situation in the Forest changes, plants tell each other about it, and the Song changes. Messages about small things only go small distances, but if something big happens, the whole Forest will know. Elves will pick up the changes in the Song quicker than most other plants and they will react to it, taking what actions are necessary for the well-being of the Forest. The other 'mode' of Elfsense is the specific examination of the Song of one plant in an attempt to learn why it sings in such a way. It's like a human looking carefully at a specific thing, and is the use of Elfsense described in ''Elder Secrets''. So what does this tell us about elven psychology? Two things, mainly. The first stems from the fact that elves perceive the world primarily through their Elfsense. If they go somewhere with less plant life, the Song will be attenuated and the elf will become deprived of sensory input. The equivalent for a human would be a deadening of all sense of their own body. No sense of touch, heat, cold, hunger, balance. No elf in their right mind voluntarily leaves the Forest. Which means that most of the elves seen outside their Forests are not in their right minds. They are either monomaniacal fanatics, prepared to suffer terrible privations for the good of the Forest (similar the Krjalki elves of Hellwood, embracing Chaos in order to fight it), or they are actually quite mad. Sometimes, right-thinking elves do leave the Forest on the business of the Forest, but this only happens in extremis, and it is very unpleasant for the elves. This stress on the elf can often be enough to temporarily unbalance the individual, mainly because it sees itself as an individual, often for the first time. The second result of Elfsense is that elves cannot really give reasons for their actions. If a human is thirsty, they will reach out for a glass of water and drink it. An elf would understand this as a metaphor for their actions, but instead of seeing themselves in the place of the human, they see themselves as the hand, and the Forest as the organism that is thirsty. An elf can no more explain its actions than a person's hand can explain why it grasps the glass. If the Forest has a need which can only be satisfied by sending its elves outside itself, the elves will go, but they won't know why. All the elves hear is the Song of the Forest expressing a need to fetch that glass of water. If the elf is asked why it needs the glass of water it can't answer. And as humans only ever see the elves, it is little wonder that elves have acquired a reputation for being callous, arbitrary, and unfriendly. Of course, the elf can use all their gargantuan intellect to find the best way to fetch that glass of water, and this plan may be convoluted indeed, but they can't explain they must fetch something that is of no direct use to themselves. Finally, the role of Elfsense explains the existence of Rootless and Renegade elves. Just as some humans are blind, some elves cannot perceive the Song of the Forest. This disability means that the afflicted elves cannot really operate in mainstream Aldryami society as Aldrya, the Forest, does not have the same immediate presence in their lives. In most cases, the Forest will nurture the unsensing elf and encourage them the take the fullest part they can in the Forest. This often takes the form of worshipping other gods, acting as messengers to other places and races, and sometimes diplomats. These are needed roles and the elves that fill them are both pitied and praised. Occasionally, the isolation the Rootless elf feels becomes too much and they become Renegade, turning their back on the Forest completely, and sating their own base desires. These are twisted individuals indeed. [[Category:Glorantha]] e31b9de924167336f25acd293ebaba3b0779bfdc 14 1459 1633 1586 2008-01-13T21:24:21Z Neil 2 wikitext text/x-wiki Here are some general thoughts and essays on Glorantha, some rules extensions for RuneQuest, and thoughts on how to run Heroquests. There is more information on Prax, developed for the [[:Category:Yellowtail|Yellowtail campaign]], and the Tertorae clan of the [[:Category:Cinsina|Cinsina tribe]] of Sartar. Some of my writings have been published by the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word], a Gloranthan fanzine and independent publishing house. [http://www.celtic-webs.com/theunspokenword/books/rushbachelors.html Rush's Bachelors] are a hero band that didn't make it into the Thieves' Arm issue. On my page, you can find some [[Furthest Shadow War extras|extra snippets for the Furthest Shadow War]], the climax scenario in the Thieves' Arm. [[Category:RPGs]] ea6c27b1ad7e61c14800e8d7beff4830cba03317 1656 1633 2008-02-03T11:54:37Z Neil 2 wikitext text/x-wiki Here are some general thoughts and essays on Glorantha, some rules extensions for RuneQuest, and thoughts on how to run Heroquests. There is more information on Prax, developed for the [[:Category:Yellowtail|Yellowtail campaign]], and the Tertorae clan of the [[:Category:Cinsina|Cinsina tribe]] of Sartar. Some of my writings have been published by the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word], a Gloranthan fanzine and independent publishing house. [http://www.celtic-webs.com/theunspokenword/books/rushbachelors.html Rush's Bachelors] are a hero band that didn't make it into the Thieves' Arm issue. On my page, you can find some [[Furthest Shadow War extras|extra snippets for the Furthest Shadow War]], the climax scenario in the Thieves' Arm. <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] 4fb41bb2c64f6fc75e2160d5d32785e1e4c77cc1 1658 1656 2008-02-03T12:00:33Z Neil 2 wikitext text/x-wiki Here are some general thoughts and essays on Glorantha, and thoughts on how to run Heroquests. There is more information on Prax, developed for the [[:Category:Yellowtail|Yellowtail campaign]], and the [[:Category:Tertorae|Tertorae clan]] of the Cinsina tribe of Sartar. Some of my writings have been published by the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word], a Gloranthan fanzine and independent publishing house. [http://www.celtic-webs.com/theunspokenword/books/rushbachelors.html Rush's Bachelors] are a hero band that didn't make it into the Thieves' Arm issue. On my page, you can find some [[Furthest Shadow War extras|extra snippets for the Furthest Shadow War]], the climax scenario in the Thieves' Arm. <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] 2b8c75ca4651160dd352d45c54449de967f73950 1661 1658 2008-02-03T12:46:32Z Neil 2 wikitext text/x-wiki Here are some general thoughts and essays on Glorantha, and thoughts on how to run Heroquests. There is more information on Prax, developed for the [[:Category:Yellowtail|Yellowtail campaign]], and the [[:Category:Tertorae|Tertorae clan]] of the Cinsina tribe of Sartar. Some of my writings have been published by the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word], a Gloranthan fanzine and independent publishing house. [http://www.celtic-webs.com/theunspokenword/books/rushbachelors.html Rush's Bachelors] are a hero band that didn't make it into the Thieves' Arm issue. On my page, you can find some [[Furthest Shadow War extras|extra snippets for the Furthest Shadow War]], the climax scenario in the Thieves' Arm. I also have a [[HeroQuest summary tables|summary of the HeroQuest conflict resolution tables]] and some [[HeroQuest narrator tips|tips for narrators]]. <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] 5b127ff9fc253aacda9c33071c3c80487935ff13 1665 1661 2008-02-08T14:22:35Z Neil 2 wikitext text/x-wiki Here are some general thoughts and essays on Glorantha, and thoughts on how to run Heroquests. There is more information on Prax, developed for the [[:Category:Yellowtail|Yellowtail campaign]], and the [[:Category:Tertorae|Tertorae clan]] of the Cinsina tribe of Sartar. Some of my writings have been published by the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word], a Gloranthan fanzine and independent publishing house. [http://www.celtic-webs.com/theunspokenword/books/rushbachelors.html Rush's Bachelors] are a hero band that didn't make it into the Thieves' Arm issue. On my page, you can find some [[Furthest Shadow War extras|extra snippets for the Furthest Shadow War]], the climax scenario in the Thieves' Arm. My [[:Category:HeroQuest|HeroQuest]] pages have a [[HeroQuest summary tables|summary of the HeroQuest conflict resolution tables]] and some [[HeroQuest narrator tips|tips for narrators]]. <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] d3322a7c014179267febee5662e438edd35e167d 0 1490 1634 2008-01-13T21:32:32Z Neil 2 wikitext text/x-wiki Here are a couple of extras for the Furthest Shadow War (as published in the Thieves' Arm from the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word]) that fell out due to space limits. I hope you find them of some interest. == Gang Interaction Matrix == The Furthest Shadow War involves some politicking among the various gangs of the Furthest underworld. This matrix gives a guide to how the gangs feel about each other. Each cell shows how the row gang feels about the column gang, e.g. Cadoric gang have no real feelings towards the Bratvar, while the Bratvar are publicly friendly to the Cadoric gang (though they are secretly enemies). {| border="1" | || '''Brat''' || '''ESC''' || '''Cad''' || '''Temp''' || '''Old''' || '''Big''' || '''Gangs''' || '''LG''' |- | '''Brat''' || || -! || +(-!) || - || +(-!) || N || - || -! |- | '''ESC''' || -! || || -(N) || - || -(+!) || * || - || +! |} Cad N N N N N N - Temp +(-!) N(-) -! +(-) N - -! Old N(-!) -(-!) N + - - -(+!) Big + N - +(-) - N - Gangs N N N - + - -! LG -(-!) +!(-) -(N) - N(+) -(N) -(N) Key + Likes +! Ally - Dislikes -! Enemy * Haven't met N No opinion ( ) Secretly... Brat Bratvar ESC East-Side Clan Cad Cadoric Temp Tempestas Old Old Fellows Big Big End Boys Gangs Other gangs: Quiet Thunder, Bastard Twins, and Dorvan's Clan LG Lion Guard Note: the Interaction Matrix isn't my idea: I stole it from some articles by Chad Underkoffler on Pyramid. JarEel-za and Krarsht-ra The Furthest Shadow War ends with a huge battle between the avatars of JarEel and Krarsht. If you want to play it out in a bit more detail than provided in the Theive's Arm, try using these ability ratings: JarEel-za: Large 7w3, Huge Scimitar 2w5^20, Fire Beams of Lunar Light From Eyes 8w4^12, Protective Glow of Rufelza ^16, Levitation 18w4, Beautiful 12w4, Noble Lunar Heroine 15w3. Krarsht-ra: Large 18w3, Long Acidic Tongue 15w4^15, Claw and Bite 18w3^8, Spit Praztim 3w4^25, Chitin ^20, Hideous 15w4, Foul Chaos Monstrosity 15w3. cd8f36523607af6c44780e3c233b4639add1f485 1635 1634 2008-01-13T21:45:56Z Neil 2 wikitext text/x-wiki Here are a couple of extras for the Furthest Shadow War (as published in the Thieves' Arm from the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word]) that fell out due to space limits. I hope you find them of some interest. == Gang Interaction Matrix == The Furthest Shadow War involves some politicking among the various gangs of the Furthest underworld. This matrix gives a guide to how the gangs feel about each other. Each cell shows how the row gang feels about the column gang, e.g. Cadoric gang have no real feelings towards the Bratvar, while the Bratvar are publicly friendly to the Cadoric gang (though they are secretly enemies). {| border="1" | || '''Brat''' || '''ESC''' || '''Cad''' || '''Temp''' || '''Old''' || '''Big''' || '''Gangs''' || '''LG''' |- | '''Brat''' || || -! || +(-!) || - || +(-!) || N || - || -! |- | '''ESC''' || -! || || -(N) || - || -(+!) || * || - || +! |- | '''Cad''' || N || N || || N || N || N || N || - |- | '''Temp''' || +(-!) || N(-) || -! || || +(-) || N || - || -! |- | '''Old''' || N(-!) || -(-!) || N || + || || - || - || -(+!) |- | '''Big''' || + || N || - || +(-) || - || || N || - |- | '''Gangs''' || N || N || N || - || + || - || || -! |- | '''LG''' || -(-!) || +!(-) || -(N) || - || N(+) || -(N) || -(N) || |} === Key === {| border="1" | {| | + || Likes |- | +! || Ally |- | - || Dislikes |- | -! || Enemy |- | * || Haven't met |- | N || No opinion |- | ( ) || Secretly... |} | {| | Brat || Bratvar |- | ESC || East-Side Clan |- | Cad || Cadoric |- | Temp || Tempestas |- | Old || Old Fellows |- | Big || Big End Boys |- | Gangs || Other gangs: Quiet Thunder, Bastard Twins, and Dorvan's Clan |- | LG || Lion Guard |} |} :Note: the Interaction Matrix isn't my idea: I stole it from some articles by Chad Underkoffler on Pyramid. == JarEel-za and Krarsht-ra == The Furthest Shadow War ends with a huge battle between the avatars of JarEel and Krarsht. If you want to play it out in a bit more detail than provided in the Theive's Arm, try using these ability ratings: :'''JarEel-za''': Large 7w3, Huge Scimitar 2w5^20, Fire Beams of Lunar Light From Eyes 8w4^12, Protective Glow of Rufelza ^16, Levitation 18w4, Beautiful 12w4, Noble Lunar Heroine 15w3. :'''Krarsht-ra''': Large 18w3, Long Acidic Tongue 15w4^15, Claw and Bite 18w3^8, Spit Praztim 3w4^25, Chitin ^20, Hideous 15w4, Foul Chaos Monstrosity 15w3. b067f6b3d3f859d1d3693c0cc9af1bf240edd9d2 1636 1635 2008-01-13T21:46:32Z Neil 2 wikitext text/x-wiki Here are a couple of extras for the Furthest Shadow War (as published in the Thieves' Arm from the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word]) that fell out due to space limits. I hope you find them of some interest. == Gang Interaction Matrix == The Furthest Shadow War involves some politicking among the various gangs of the Furthest underworld. This matrix gives a guide to how the gangs feel about each other. Each cell shows how the row gang feels about the column gang, e.g. Cadoric gang have no real feelings towards the Bratvar, while the Bratvar are publicly friendly to the Cadoric gang (though they are secretly enemies). {| border="1" | || '''Brat''' || '''ESC''' || '''Cad''' || '''Temp''' || '''Old''' || '''Big''' || '''Gangs''' || '''LG''' |- | '''Brat''' || || -! || +(-!) || - || +(-!) || N || - || -! |- | '''ESC''' || -! || || -(N) || - || -(+!) || * || - || +! |- | '''Cad''' || N || N || || N || N || N || N || - |- | '''Temp''' || +(-!) || N(-) || -! || || +(-) || N || - || -! |- | '''Old''' || N(-!) || -(-!) || N || + || || - || - || -(+!) |- | '''Big''' || + || N || - || +(-) || - || || N || - |- | '''Gangs''' || N || N || N || - || + || - || || -! |- | '''LG''' || -(-!) || +!(-) || -(N) || - || N(+) || -(N) || -(N) || |} === Key === {| border="1" | {| | + || Likes |- | +! || Ally |- | - || Dislikes |- | -! || Enemy |- | * || Haven't met |- | N || No opinion |- | ( ) || Secretly... |} | {| | Brat || Bratvar |- | ESC || East-Side Clan |- | Cad || Cadoric |- | Temp || Tempestas |- | Old || Old Fellows |- | Big || Big End Boys |- | Gangs || Other gangs: Quiet Thunder, Bastard Twins, and Dorvan's Clan |- | LG || Lion Guard |} |} :Note: the Interaction Matrix isn't my idea: I stole it from some articles by Chad Underkoffler on Pyramid. == JarEel-za and Krarsht-ra == The Furthest Shadow War ends with a huge battle between the avatars of JarEel and Krarsht. If you want to play it out in a bit more detail than provided in the Theive's Arm, try using these ability ratings: :'''JarEel-za''': Large 7w3, Huge Scimitar 2w5^20, Fire Beams of Lunar Light From Eyes 8w4^12, Protective Glow of Rufelza ^16, Levitation 18w4, Beautiful 12w4, Noble Lunar Heroine 15w3. :'''Krarsht-ra''': Large 18w3, Long Acidic Tongue 15w4^15, Claw and Bite 18w3^8, Spit Praztim 3w4^25, Chitin ^20, Hideous 15w4, Foul Chaos Monstrosity 15w3. [[Category:RPGs]] 48bac4c0b0aad8770d7406c5c23c4e214f25c262 1637 1636 2008-01-13T21:47:04Z Neil 2 wikitext text/x-wiki Here are a couple of extras for the Furthest Shadow War (as published in the Thieves' Arm from the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word]) that fell out due to space limits. I hope you find them of some interest. == Gang Interaction Matrix == The Furthest Shadow War involves some politicking among the various gangs of the Furthest underworld. This matrix gives a guide to how the gangs feel about each other. Each cell shows how the row gang feels about the column gang, e.g. Cadoric gang have no real feelings towards the Bratvar, while the Bratvar are publicly friendly to the Cadoric gang (though they are secretly enemies). {| border="1" | || '''Brat''' || '''ESC''' || '''Cad''' || '''Temp''' || '''Old''' || '''Big''' || '''Gangs''' || '''LG''' |- | '''Brat''' || || -! || +(-!) || - || +(-!) || N || - || -! |- | '''ESC''' || -! || || -(N) || - || -(+!) || * || - || +! |- | '''Cad''' || N || N || || N || N || N || N || - |- | '''Temp''' || +(-!) || N(-) || -! || || +(-) || N || - || -! |- | '''Old''' || N(-!) || -(-!) || N || + || || - || - || -(+!) |- | '''Big''' || + || N || - || +(-) || - || || N || - |- | '''Gangs''' || N || N || N || - || + || - || || -! |- | '''LG''' || -(-!) || +!(-) || -(N) || - || N(+) || -(N) || -(N) || |} === Key === {| border="1" | {| | + || Likes |- | +! || Ally |- | - || Dislikes |- | -! || Enemy |- | * || Haven't met |- | N || No opinion |- | ( ) || Secretly... |} | {| | Brat || Bratvar |- | ESC || East-Side Clan |- | Cad || Cadoric |- | Temp || Tempestas |- | Old || Old Fellows |- | Big || Big End Boys |- | Gangs || Other gangs: Quiet Thunder, Bastard Twins, and Dorvan's Clan |- | LG || Lion Guard |} |} :Note: the Interaction Matrix isn't my idea: I stole it from some articles by Chad Underkoffler on Pyramid. == JarEel-za and Krarsht-ra == The Furthest Shadow War ends with a huge battle between the avatars of JarEel and Krarsht. If you want to play it out in a bit more detail than provided in the Theive's Arm, try using these ability ratings: :'''JarEel-za''': Large 7w3, Huge Scimitar 2w5^20, Fire Beams of Lunar Light From Eyes 8w4^12, Protective Glow of Rufelza ^16, Levitation 18w4, Beautiful 12w4, Noble Lunar Heroine 15w3. :'''Krarsht-ra''': Large 18w3, Long Acidic Tongue 15w4^15, Claw and Bite 18w3^8, Spit Praztim 3w4^25, Chitin ^20, Hideous 15w4, Foul Chaos Monstrosity 15w3. [[Category:Glorantha]] 3e83368ef2552c4b94203ce84a29574adcd9a839 0 1491 1638 2008-01-16T21:55:52Z Neil 2 wikitext text/x-wiki '''''Note:''' This information is what a typical labourer, living in the city of New Pavis at about 1615 ST (Since Time), would know about the history, geography and politics of the world about him. This information is limited by his knowledge and is coloured by his prejudices. Much of this information is incomplete, some is inaccurate, and parts are just plain wrong.'' '''''General background:''' The world is Glorantha, the continent Genertela and the area is Prax. Pavis is the only city of note in Prax. The technology is Bronze Age and magic is commonplace. People, place and things are often categorised by the god they are affiliated to and draw power from.'' '''''Currency:''' the basic unit is the silver piece, generally called a Lunar (Lunars call them 'Imperials'; die-hard Orlanthi call them 'Guilders'). One Lunar is the minimum needed for a day's food and lodging in Pavis. There are ten copper Clacks to a Lunar and twenty Lunars make a golden Wheel.'' == Geography == The River of Cradles (also known as the Zola Fel river) flows from the Troll-infested highlands of Dagori Inkath in the north, through the arid grasslands of Prax to the Rozgali Sea at Corflu. Pavis is situated in the middle of Prax on the River of Cradles. The area around Pavis, Pavis County, is ruled directly from the city. Pavis County is heavily settled and farmed, mainly by Sartarite colonists. Pavis is the jewel of Prax. People from all lands live here. Most people are simple Orlanth-worshipping crafters and farmers, descendants of the colonists that came when the Lunars conquered the land of Sartar. The Lunars control the lands of Prax from here; their forces are garrisoned in the north of the city. The insular Sun Domers control 'their' quarter of Pavis. The Ingilli family, leaders of the River People, live near their sacred river. Praxians are forbidden to camp near Pavis as they once tried to capture the city. Within the giant-built walls of Old Pavis, the ruins that are the Big Rubble are a home for many outlaws, treasure seekers, and worse. Some people live in the Manside area, farming and treasure seeking. The Zebra Riders have their grazing lands in the Rubble. These are often raided by the evil Trolls which live in the south of the Rubble. Thankfully, they are kept in check by the Elves of the Garden. Chaos abominations thrive in the Devil's Playground. Down river from Pavis is Sun Dome County, populated by Yelmalion farmers. Great fields of corn surround the river. Border controls are tight. The land surrounding the southern part of the Zola Fel is known as the Grantlands. This area was annexed by Lunar forces, and a program of organised settlement by Lunar farmers is beginning. The area is protected by Lunar troops based at Fort Ronegarth. In the delta near the mouth of the river is Corflu, the only port for hundreds of kilometres along the coast. There are several swamps along the Zola Fel. They are choked with reeds and thickets and are a favourite haunting (and hunting) ground of biting insects, outlaws and chaotic monsters. In the spaces between the swamps, the ground is fertile and fairly safe, as Praxian bandits cannot scale the cliffs either side of the river valley. To the west of the River of Cradles is the empty land of Prax. The savannah of Prax makes for good grazing, especially around the oases, and there are often disputes between Praxians trying to gain the best foraging for their herds. There is a major Praxian temple complex at the Paps oasis. The Sable nation dominates Prax now, a reward for their collaboration with the Lunars at the Battle of Moonbroth. Further west, Prax rises into the mountains of Sartar and the Holy Country; beyond them lies the magical place of Dragon Pass. East of the river, the Wastes quickly turn into desolate, inhospitable desert, but this is where most Praxians live. == History == Orlanth is King of the gods, by right of his Quest to return Yelm the Sun to life from the Underworld (that Orlanth killed Yelm in the first place is a sore point with Orlanthi). But he is being challenged by the Red Goddess, patron of the Lunar Empire. The Red Goddess lived many hundreds of years ago and succeeded in her HeroQuest to be born again as the Red Goddess, and she now resides in the Middle Air as the moon, her changing face visible in the far north-west. Because she was born after the Great Compromise, her followers have accommodated all things in the world, including Chaos. Pavis was an half-elf Hero of the Second Age. With the help of Giants, he built Old Pavis and struck deals with dwarves and elves. Eventually the Trolls, led by Gerak Kag, jumped over the walls of the old city and sacked it. This made the Big Rubble and the treasures of Pavis can still be found there. After many hundreds of years, settlers came from Sartar and founded New Pavis, or Pavis Outside The Walls, in 1550. Some 40 years later, the Lunars invaded Sartar. After the Lunars had been defeated in the Holy Country, they came to Prax. They defeated the Praxian nomads at the battle of Moonbroth and took Pavis without resistance. That was five years ago, in 1610. Some of the army of occupation has returned home, but there are many soldiers patrolling New Pavis, the Big Rubble and the rest of Prax. == Cultures == ;Old Pavic :These are the people that have lived in Pavis and the Rubble for generations. While most Pavisites live in New Pavis, there is still a large number living in Manside in the Rubble. They speak mainly Pavic, though some speak Old Pavic. ;New Pavic :These are Orlanthi people whose ancestors lived in the mountains of Sartar. Many came here years ago to escape the Lunar conquest of Sartar. All men are expected to be warriors as well as farmers or herders. They speak Pavic or Sartarite. They are a proud people, and resent the Lunars for the suppression of Orlanth. This sometimes takes the form of attacks by the Cold Wind terrorists. ;Lunar :The followers of the Red Goddess and the Lunar Way, these people have a great empire in the far north west. They conquered Prax and Pavis five years ago. They have brought with them many laws and officials, but they have brought order and safety to the area. The Grantlands are being settled by Lunar farmers, under the protection of Duke Raus. Lunars speak New Pelorian. ;Praxian :The Praxians are nomads that roam across all of Prax. They are separated into Tribes according to what animals they herd and eat. The Five Great Tribes are Impala, High Llama, Sable, Morokanth and Bison. The lesser tribes are many and include Unicorn, Bolo Lizard, Zebra, and Rhino. To Praxians, horses are unclean and must not be touched, even to eat. Praxians come to Pavis to trade their animals for metal goods, though they prefer to steal. The exception is the Zebra Tribe which live in the Big Rubble. All tribes speak dialects of Praxian. ;River People :These are the fishermen and boat people. They are mainly newtlings, but there are a sizeable number of humans as well. They live in small villages scattered the whole length of the Zola Fel. Often criminals join the River People to avoid justice. They speak Boatspeech, and some speak Pavic or Tradetalk. ;Oasis People :These introverted people have lived near the desert oases since the Godtime. No-one knows either their origin or their language. They have no warriors and many are enslaved by the Praxians. The Lunars have started to act against this practice. ;Sun Domers :The community of Sun worshippers that came to Prax when Pavis lived. They control a large area of land and rule it with a rod of Iron. Mercenary groups are often hired out, and a small number of Sun Domers lives in New Pavis. Sun Domers speak Pavic with Firespeech for formal and religious ceremonies. Sun Domers prefer to trade using Wheels, as gold is holy to them. == Races == ;Human :Humans are by far the most commonplace people in Prax and have spawned many cultures. ;Troll :Creatures of the night, trolls are voracious eaters and cunning hunters. Many trolls live in the south of the Big Rubble. There are different types of troll, including great trolls and the child-sized trollkin. Trollkin are often used as food. ;Duck :In the Godtime, some people revered their birds more than the gods, so they were cursed to become flightless birds. Ducks are good swimmers and are often found in fishing communities. They have linked to Orlanthi terrorist groups and have been declared outlaw by the Lunars. ;Agamor :Descended from Fire, Agamori (also known as Men-and-a-Half) are very tall and strong. They form one of the minor tribes of Prax. ;Newtling :These child-sized amphibian creatures are common in the Zola Fel. They carry out much trade and fishing. Newtling tail is considered a great delicacy in some circles. ;Morokanth :A race of intelligent beasts, these Praxians look like tapirs and weigh as much as two men. They herd men as food animals. ;Broo :A race of chaos beastmen, they spread destruction and pestilence wherever they go. The Lunars have put a bounty on broo heads. ;Baboon :These are giant monkeys that have captured the Man Rune. They are dirty, disorganised and a major nuisance in the wilder parts of Prax and the Big Rubble. ;Elf :Dwellers of forests, they kill anything and anyone venturing into their forests without permission. They have little other contact with the outside world. A group of elves lives in the Garden in the south of the Big Rubble, brought there by Pavis Half-Elven. ;Dwarf :Also called Mostali, they are very secretive. They are great inventors (the crossbow is a dwarven invention) and masons (they built both Old and New Pavis). A group of dwarves lives in tunnels under the Big Rubble and New Pavis. ;Ogre :A race of chaotic, cannibalistic humans, they often pass unnoticed among us, until they murder. == Gods and religions == Most religions (cults) have several stages of membership. Lay worshippers have little commitment to the god and receive little in the way of benefit. However, most people are Initiates; they have given part of their soul to the god, and generally give 1/10 of their time and income (often in kind) to the temple. They must worship once a week, give Power to their god on holy days and keep the cult strictures. In return, they are taught spirit magic by the priests, can sacrifice for divine magic and often receive mundane training as well. Initiates also have the ability to call for Divine Intervention. Priests are the spiritual leaders of the people. They live for the god and receive great magical powers. Some cults (especially warlike ones) also have Lords. These are people that have achieved great skill, often in arms, and devote themselves to working for their god. === Gods of the Zola Fel valley === ;Pavis :The god that was Pavis Half-Elven and is now the spirit of the city of Pavis. The Sons and Daughters (mostly male) of Pavis form the ruling council of the city. ;Flintnail :Half-dwarven friend of Pavis, Flintnail is a god of dwarves and stonemasons. His worshippers strive to restore the Big Rubble to its former glory. ;Zola Fel :Goddess of the River of Cradles, she is worshipped by fish, newtlings and some fishermen. ;Yelmalio :Son of Yelm the Sun, Yelmalion people are stoic, taciturn and insular. Their communities are very strict and their contact with the rest of the world is often limited to the large mercenary forces they hire out. Yelmalions specialise in the use of Sky weapons: spear, javelin and bow. === Orlanthi Gods (Lightbringers) === ;Orlanth :King of Gods, God of Storms, Orlanth is loud, violent, and fickle. Chief god of the Satarite pantheon, he is the enemy of the Red Goddess and his worship is discouraged by the Lunars. Many instead worship Barntar, Orlanth's farmer aspect. ;Ernalda :Wife of Orlanth, Earth Mother and Grain Goddess, she is worshipped by most farmer women. ;Issaries :God of Trade and Communication, he is worshipped by merchants and traders. Non-violent, Issaries prefers to build trust between peoples (more profits that way!). Inventor of Tradetalk. ;Lhankor Mhy :God of Knowledge and Lawspeaker. Lhankor Mhy priests (also called Sages) seek knowledge, which they sell. Sages act as legal adjudicators, as well as being sought after as scribes, translators, and experts in various fields. ;Chalana Arroy :Goddess of Healing, she will cure anyone of their ills -- for a price. Once she has extended her protection over someone, no-one may harm them else they will be cursed to not receive the services of a healer again. Her followers wear a white sash, do not carry weapons and may not harm any living being. ;Humakt :Discoverer of Death, Humakt was the first to kill. Humakti worship death; once killed, they cannot be resurrected. They are also honourable to the extreme and Humakt often places geasa on his followers to make them better Humakti. Humakti favour the sword. === Lunar Gods === ;Seven Mothers :The Lunar missionary cult, the Seven Mothers are the seven Heroes that brought the Red Goddess to godhood, and in doing so healed themselves. The missionaries of the Seven Mothers spread the Lunar Way of acceptance to those on the borders of the empire. ;Yanafal Tarnils :The Lunar war and death god. Worshipped by Lunar soldiers, he has been healed of Humakt's self-destructive behaviour and allows resurrection. ;Etyries :The trader and traveller, she is the patron of Lunar traders, emissaries, and propagandists. The Etyries cult now controls the markets in Pavis. ;Irrippi Ontor :The sages of Irrippi Ontor are always searching for new knowledge wherever it may be found and give weekly lectures on what they have discovered. They also control the complex Lunar bureaucracy. ;Teelo Norri :Protector of the weak and champion of children. There is a Teelo Norri orphanage in Pavis, open to all homeless children, and a soup kitchen. === Praxian Gods === ;Waha :Father of Beasts, he is the hunter and butcher that taught people to eat beasts. The majority of Praxian men worship him. He teaches the Peaceful Cut, a ritual butchery skill, but prevents his followers knowing healing magic. ;Eiritha :The Herd Mother, she is responsible for life and growth in the Praxian herds. ;Storm Bull :Berserk god of Death and Rage, followers of Urox are crass, loud, violent and are pledged to destroy chaos wherever it may be found. ;Daka Fal :The god of the dead. Followers of Daka Fal summon their ancestors for wisdom, guidance and power. === Non-Human Gods === ;Kyger Litor :Goddess of Darkness and Mother of all Trolls. ;Zorak Zoran :Troll god of Death and Destruction. Zorak Zorani often fight on after death as zombies or ghosts. ;Argan Argar :Troll God of Commerce and Surface Darkness. ;Aldrya :Goddess of Elves and forests, she is very protective and few intruders return from elven woods. ;Mostal :Father of the dwarven race. Dwarfs refuse to discuss him, like most other things. ;The God of Five Thousand Eyes :Newtling god. Nothing is known of this god by humans. === Chaos Gods === ;Malia :Goddess of Disease, her agents are the hated Broos. ;Thanatar :God of the Severed Head, Stealer of Knowledge and Souls. ;Krarsht :The Maw Beneath, cult of hideous burrowing and tunnelling creatures. ;Wakboth :The Devil, trapped by the Storm Bull under the Block in Prax. ;Primal Chaos :The Ooze; the slime of primeval Chaos. == Cultural Stereotypes == === Pavic attitudes to Lunars === While we would rather be masters of our own fates than be ruled by the Lunars, they are not evil like the Sartarites say. Their laws may be harsh, but they work and they have brought some law to the plains of Prax. Also, their army needs supplies, for which they pay generously in silver and jobs. === Praxian attitudes to Lunars and all farmers === Lunars are horse-riding wimps that live in immobile stone tents. As such, they are beneath contempt. Their caravans are good to raid though, providing they are not too strong. The farmers settle and grow food which is good for taking.. They can also be traded with, so that we may have the metal weapons needed to show our bravery. === Lunar attitudes to Pavisites/Praxians === The Red Goddess, in her benevolence, has sent us here to this barren and goddess-forsaken land to pacify it with the true words and thoughts of the Lunar Way. The barbarians here need to be educated and civilised, that they might become good, upstanding citizens of the Lunar Empire. All Hail the Reaching Moon! [[Category:Yellowtail]] [[Category:Glorantha]] 0d8d1e4b6fbadff9d309002bb57b296a19c29462 0 1492 1639 2008-01-16T23:18:14Z Neil 2 wikitext text/x-wiki ''This is the story of the coming-of-age ritual of my current PC, Ozur Dragonfriend. The campaign is based in a variant Glorantha. The local area is a large archipelago and Gont is one of the smaller islands. Gont is also the name of the mountain in the centre of the island and the name of the god that lives there. The culture is pretty much Orlanthi barbarian. Gontish children become adults at about 15 by completing a quest on the mountain of Gont. No adult speaks to the children of what is involved, but the adult takes their name from an unusual happening during that quest. The story is written as Dragonfriend would tell it to the other adults one evening on Gont (assuming, of course, that he lives that long!).'' I started my quest for my true name on the sacred day of Orlanth the Adventurer [1], Windsday of Movement week in Storm season. The day dawned clear and cold, with a layer of snow covering the land. Soon after dawn I was summoned to Broken Cup's hearth [2]. There he greeted me and gave me kind words about my upcoming quest. As we parted, he gave me a viridian cloak [3] to keep me warm on my quest and after. Then his son, Wolfleap, led me to the circle of standing stones that is sacred to Gont. I had never been beyond this point, as I was still a child. Once there, Wolfleap explained my task to me. I was to find five things: some snow, some red earth, a small white and blue flower, some food, and my true name [4]. These were to be retrieved from the mountain of Gont. With that, he left and I started on my journey up Gont. ---- I resolved to first reach the summit of Gont, where I would gain a clear view of the island that is my home, communion with Gont and a clearer insight into the world of Orlanth, king of Gods and Storms. I spent much of Windsday climbing Gont. While I climbed I searched and I found the first of my quest items, some red earth. By nightfall, I was near the top of Gont. I found a place to camp and built my fire there. That night, I dreamt of my fire and of my return to the Great Steading [5]. I was woken near midnight by the sound of a wolf howling. I peered into the night and saw the wolf. It was pure white, and came towards my fire, where it sat and warmed itself, watching me without malice. After a few moments, thinking I might befriend this animal, I threw it some of my meat. The wolf ate this and then walked off into the night. Perhaps this was the wolf that gave Wolfleap his name? With this thought in my mind, I soon slept again. ---- I awoke when Yelm disturbed my slumber. My fire was still burning, though I had not fed it well. I gave thanks to Gont for this gift of warmth and headed up the mountain again. Just after noon I reached the summit of Gont. I could see forever in the warm and still air. Not even the snow was cold to the touch. All of Gont the Island was below me, open to my gaze. I gave thanks to Gont and to Orlanth for my safe passage to this place and I then proceeded to search for this elusive flower. As I dug in the snow, I became aware of another of Gont's gifts. The warm snow of Gont rubbed my hand as I searched and healed a small burn on my hand from the fire the night before. I praised Gont for this extra gift and resolved to take some of this healing snow down with me. I searched for a little while and soon found the flower I was looking for. Tired from my labours and the climb, I lay back in the healing snow and drifted off to sleep. While I slept, I had another dream. This dream was not a normal dream, but seemed almost as real as being awake. In this dream, a strange creature approached me. It was very tall, standing head and shoulders above Flowerhair, the tallest man on Gont. I recognised it from the skalds' tales as a dragonewt, though this creature was different to those described in the stories. The dragonewt did not speak, but it approached me in a spirit of friendship. The dragonewt guided my dream self down the mountain and back to the stone circle where I had started my journey. Once there, the creature stood on the altarstone and raised its hands above its head until they met. Then I awoke. After collecting some of the healing snow, it only remained for me to find some food and return to the Steading. I followed the path the dragonewt had shown me and I was most of the way down Gont by the time I had to make my camp. I slept without dreaming that night, though the while wolf again visited my fire. I offered it food again, and it took meat from my hand. then it left, and I slept again. ---- In the morning, I continued on the path shown to me by the dragonewt. On the way, I found a bird nesting in a tree and slew it, to complete my quest and to honour Orlanth. I soon arrived back at the starting place of my quest and, like my dream-guide, I stood on the altar and raised my hands. At the instant they touched above my head, a might magic appeared about me. There was a great roaring and rushing of wind and I was propelled across the world to a place I know not where. I arrived at a great stone pyramid. It was ancient and made of huge dressed blocks, bigger than a man, that fitted together so well that I could barely see the joins between them. I was propelled inside the huge black structure and I met my dragonewt guide again. The guide played that role again and took me through all the myriad rooms, corridors and spaces of the pyramid. All was empty and without any sign of life and I felt a great sadness in my guide. I also became aware that it was my destiny to help bring this ancient and holy place back to life. Then my spirit returned to my body on Gont. After clearing my mind as best I could, I performed the final act of my quest. I placed the red earth on the altarstone and planted the sacred flower in it. I watered it with melted snow and fertilised it with the blood of the bird I had slain. I offered all this to Gont, with thanks for his bounty, and Gont replied that he was pleased with me. When I approached the steading, Wolfleap greeted me with the welcome that is given to a stranger, as he had never before seen the man that I now was. I gave him my name of Dragonfriend and I was escorted with joy to Broken Cup's house. Broken Cup took me inside and told me of a strange happening. At the time of my journey to the pyramid, Broken Cup was visited by the ghost of my dragonewt guide which had presented him with a scroll and indicated that I was to have it. On the scroll was a map of an unknown place, with writing in a strange hand. This map was to guide me in my second journey to the dragonewt's pyramid and to many great adventures. But that is a story for another night. ---- [1] In case you don't notice, Dragonfriend is a bit of an Orlanth wannabe. [2] Broken Cup is the headman of the Gontish people. [3] Viridian is a much sought after mineral dye, found only on Gont. It is a blue-green colour and is sometimes referred to as Gont's Blood. [4] Dragonfriend is never told the reasons why these items are chosen, though he realises why by the end of the quest. [5] Only a dozen or so huts, but the biggest settlement on Gont. [[Category:Glorantha|Ozur]] d109a503c085469417677a214f1410020cbfde75 0 1493 1640 2008-01-21T20:43:13Z Neil 2 wikitext text/x-wiki People are arguing about how Chaos is perceived by Gloranthans. But I think it's useful to consider how Chaos is considered by us, the people that create Glorantha the game world. From that point of view, I think that Chaos is bad because Chaos is boring. The things that attracted me, as a player and GM, to Glorantha were its depth and the lack of easy moral decisions (Are the Lunars evil? Were the Romans evil?). This meant that I could imagine world as rich and complex as this one, and use it as a vehicle in which to tell engaging and satisfying stories. The presence of Chaos acts against this. There are several reasons for this: * Chaos is a cop-out for plot writers; * Chaos ensures players don't have to make moral or ethical decisions; * Chaos makes characters lack depth; * Chaotic villains are treated as victims; * Chaotics make for second-rate antagonists. == Chaos is a cop-out for plot writers == It's very easy to write a plot with obvious "Good Guys" (the PCs) and "Bad Guys" (the chaotics). It gives the PCs a good motivation. Chaos has nifty "weird stuff" for a challenge. But doing so cheapens Glorantha. Good drama needs well-rounded characters, with both good and evil aspects to their personalities. They need complexity, and the ability to change and take on different roles at different times. Turning everything into two-dimensional "Good vs Evil" prevents that happening. Very often, conflicts are not caused by differences between Good and Evil, but simply by different sets of people having different agendas. Giving the opposition believable reasons for their actions allows me as a GM to run those characters better and more easily. This trend towards the "Good vs Evil" cop-out has been developing "recently". As far as I know, some 17% of Chaosium-published RQ2 scenarios revolved around chaos. Some 40% of Chaosium/AH-published RQ3 scenarios did. This is a dangerous trend, turning adventures into simple munchkin-friendly hack'n'slay sessions. Glorantha can be so much more than that. This trend might be reversed with the greater emphasis on socially-situated 'adventures' that might follow from the publication of Hero Wars and HeroQuest. I wait in hope. == Chaos means no moral or ethical decisions == The assumption that chaos = evil means that characters, and therefore players, never have to consider the ethics of their actions. For example, at the end of Gamauta's Vision, the PCs have to decide what to do with a bunch of chaotic children. An interesting moral question. Lots of MGF there as the PCs explore and argue the options. At least, there would be if it wasn't obvious to the PCs that chaos = evil, therefore the children must be killed. At the beginning of the Troubled Waters campaign, the PCs had real qualms about slaughtering the cute, innocent, little mudsharks. At the end of the campaign, they had no problem killing cute, innocent, little chaos monstrosities. Guess which episode they talk about more? And removing the big "Beware: Evil! Soul Destruction in Progress!" sign from cults like Thanatar makes them more appealing, more seductive, more interesting. Humakt offers great power, but at a cost to society. Thanatar offers great power, but at a cost to society. Is one better than the other? Are there situations where gaining Thanatari or Humakti powers are justified? Does the end ever justify the means? Without chaos, you can ask this question. == Chaos makes characters lack depth == If the focus of a campaign is simply killing chaos, all the PCs will be concerned with is killing chaos. The characters will be one-dimensional. Their opponents will be chaos, and exist simply to be killed (eventually). Also one-dimensional. But give all the characters, PCs and NPCs, more objectives, more avenues, and you'll get a more rewarding and enjoyable game. == Chaotic villains are treated as victims == At present, players have no reason to sympathise with their enemies. If a character is evil, does evil things, it's not their fault. They're chaotic: they can't help it. There's no sense of people being responsible for their actions, and the effects they have. Nobody has to think why evil happens, how it can be stopped, what role they play in bringing it about. == Chaotics make for second-rate antagonists == :"Humans are the only race more dangerous that broos in the [Zola Fel] Valley" :(RoC, p. 109) If I want to give my players a run for their money, why should I give them second-rate opposition? Why not set them against the most dangerous opposition there is? In fact, I think that the presence of Chaos in Glorantha is so pernicious that I think it should be removed from all out contacts with the world. If we do that, what do we loose? Very little, if anything. * Illumination remains, as a theist version of mysticism. It would have exactly the same game effects. * Storm Bulls remain psychopathic killers, but instead of being anti-chaos, they become anti-everyone. This raises more interesting questions for their clan leaders, increasing MGF and leading to a better game. * The Orlanthi will still hate the Lunars: the Lunars have invaded, are changing the Orlanthi way of life, changing the mythic landscape. * "Chaos" gods remain hated or feared. Thanatari still steal knowledge, Krarshti still conspire, Thedi still rape. Humakti still kill. * The lesser and greater darkness from the godtime remain, but are the same thing. Very few of the myths change. * The Compromise remains, to include and control the changes of the Storm Age into the stability of the Golden Age. The real world doesn't have clearly labelled evil. This makes for difficult judgements. Gameworlds like ''Call of Cthulhu'' don't have clearly labelled evil. That makes for interesting play. Why should Glorantha be so impoverished? (For some related thoughts on morality and RPGs, have a look at John Tynes's game [http://johntynes.com/revland2000/rl_powerkill.html Power Kill].) [[Category:Glorantha]] cd94e57b2e2d9aca332f77b394eadca0855e722c 1641 1640 2008-01-21T20:46:57Z Neil 2 wikitext text/x-wiki People are arguing about how Chaos is perceived by Gloranthans. But I think it's useful to consider how Chaos is considered by us, the people that create Glorantha the game world. From that point of view, I think that Chaos is bad because Chaos is boring. The things that attracted me, as a player and GM, to Glorantha were its depth and the lack of easy moral decisions (Are the Lunars evil? Were the Romans evil?). This meant that I could imagine world as rich and complex as this one, and use it as a vehicle in which to tell engaging and satisfying stories. The presence of Chaos acts against this. There are several reasons for this. __TOC__ == Chaos is a cop-out for plot writers == It's very easy to write a plot with obvious "Good Guys" (the PCs) and "Bad Guys" (the chaotics). It gives the PCs a good motivation. Chaos has nifty "weird stuff" for a challenge. But doing so cheapens Glorantha. Good drama needs well-rounded characters, with both good and evil aspects to their personalities. They need complexity, and the ability to change and take on different roles at different times. Turning everything into two-dimensional "Good vs Evil" prevents that happening. Very often, conflicts are not caused by differences between Good and Evil, but simply by different sets of people having different agendas. Giving the opposition believable reasons for their actions allows me as a GM to run those characters better and more easily. This trend towards the "Good vs Evil" cop-out has been developing "recently". As far as I know, some 17% of Chaosium-published RQ2 scenarios revolved around chaos. Some 40% of Chaosium/AH-published RQ3 scenarios did. This is a dangerous trend, turning adventures into simple munchkin-friendly hack'n'slay sessions. Glorantha can be so much more than that. This trend might be reversed with the greater emphasis on socially-situated 'adventures' that might follow from the publication of Hero Wars and HeroQuest. I wait in hope. == Chaos means no moral or ethical decisions == The assumption that chaos = evil means that characters, and therefore players, never have to consider the ethics of their actions. For example, at the end of Gamauta's Vision, the PCs have to decide what to do with a bunch of chaotic children. An interesting moral question. Lots of MGF there as the PCs explore and argue the options. At least, there would be if it wasn't obvious to the PCs that chaos = evil, therefore the children must be killed. At the beginning of the Troubled Waters campaign, the PCs had real qualms about slaughtering the cute, innocent, little mudsharks. At the end of the campaign, they had no problem killing cute, innocent, little chaos monstrosities. Guess which episode they talk about more? And removing the big "Beware: Evil! Soul Destruction in Progress!" sign from cults like Thanatar makes them more appealing, more seductive, more interesting. Humakt offers great power, but at a cost to society. Thanatar offers great power, but at a cost to society. Is one better than the other? Are there situations where gaining Thanatari or Humakti powers are justified? Does the end ever justify the means? Without chaos, you can ask this question. == Chaos makes characters lack depth == If the focus of a campaign is simply killing chaos, all the PCs will be concerned with is killing chaos. The characters will be one-dimensional. Their opponents will be chaos, and exist simply to be killed (eventually). Also one-dimensional. But give all the characters, PCs and NPCs, more objectives, more avenues, and you'll get a more rewarding and enjoyable game. == Chaotic villains are treated as victims == At present, players have no reason to sympathise with their enemies. If a character is evil, does evil things, it's not their fault. They're chaotic: they can't help it. There's no sense of people being responsible for their actions, and the effects they have. Nobody has to think why evil happens, how it can be stopped, what role they play in bringing it about. == Chaotics make for second-rate antagonists == :"Humans are the only race more dangerous that broos in the [Zola Fel] Valley" :(RoC, p. 109) If I want to give my players a run for their money, why should I give them second-rate opposition? Why not set them against the most dangerous opposition there is? In fact, I think that the presence of Chaos in Glorantha is so pernicious that I think it should be removed from all out contacts with the world. If we do that, what do we loose? Very little, if anything. * Illumination remains, as a theist version of mysticism. It would have exactly the same game effects. * Storm Bulls remain psychopathic killers, but instead of being anti-chaos, they become anti-everyone. This raises more interesting questions for their clan leaders, increasing MGF and leading to a better game. * The Orlanthi will still hate the Lunars: the Lunars have invaded, are changing the Orlanthi way of life, changing the mythic landscape. * "Chaos" gods remain hated or feared. Thanatari still steal knowledge, Krarshti still conspire, Thedi still rape. Humakti still kill. * The lesser and greater darkness from the godtime remain, but are the same thing. Very few of the myths change. * The Compromise remains, to include and control the changes of the Storm Age into the stability of the Golden Age. The real world doesn't have clearly labelled evil. This makes for difficult judgements. Gameworlds like ''Call of Cthulhu'' don't have clearly labelled evil. That makes for interesting play. Why should Glorantha be so impoverished? (For some related thoughts on morality and RPGs, have a look at John Tynes's game [http://johntynes.com/revland2000/rl_powerkill.html Power Kill].) [[Category:Glorantha]] be68c627cca5666e8084c3b5d4b1c3095698b002 6 1494 1642 2008-01-30T21:45:17Z Neil 2 A photo of a real-life woodcut, in the style of Andrin Lawspeaker. wikitext text/x-wiki A photo of a real-life woodcut, in the style of Andrin Lawspeaker. 289b9a67d749c32b649e809b3e8a0b10b5751037 1643 1642 2008-01-30T21:54:46Z Neil 2 wikitext text/x-wiki A photo of a real-life woodcut, in the style of the illustration of Andrin Lawspeaker from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. e50df76fa39ac8339188ad9e511dcc4642bd1f2c 6 1495 1644 2008-01-30T21:55:13Z Neil 2 A photo of a real-life woodcut, in the style of the illustration of Andrin Lawspeaker from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. wikitext text/x-wiki A photo of a real-life woodcut, in the style of the illustration of Andrin Lawspeaker from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. e50df76fa39ac8339188ad9e511dcc4642bd1f2c 6 1496 1645 2008-01-30T21:56:32Z Neil 2 A photo of a real-life woodcut, in the style of the illustration of Orane Steadwife from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. wikitext text/x-wiki A photo of a real-life woodcut, in the style of the illustration of Orane Steadwife from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. 4d8643e5b7713ceb6f949562a0865ea85918e134 6 1497 1646 2008-01-30T21:57:20Z Neil 2 A photo of a real-life woodcut, in the style of the illustration of Pelaskos the Fisher from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. wikitext text/x-wiki A photo of a real-life woodcut, in the style of the illustration of Pelaskos the Fisher from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. 63d1c6adad7975aab24330f1697753fb4e7c6cd2 6 1498 1647 2008-01-30T21:58:04Z Neil 2 A photo of a real-life woodcut, in the style of the illustration of Pelaskos the Fisher from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. wikitext text/x-wiki A photo of a real-life woodcut, in the style of the illustration of Pelaskos the Fisher from the Hero Wars supplements [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. 63d1c6adad7975aab24330f1697753fb4e7c6cd2 0 1499 1648 2008-01-30T22:06:26Z Neil 2 wikitext text/x-wiki These are some over-life-sized wooden statues in my parents' town of Uckfield in Sussex. I'm not sure what they're really called, but they're similar in style to the woodcuts in the Orlanthi Hero Wars books, [http://www.glorantha.com/products/1301.html Thunder Rebels] and [http://www.glorantha.com/products/1310.html Storm Tribe]. === Andrin Lawspeaker === <gallery> Image:Andrin1-small.jpg Image:Andrin2-small.jpg </gallery> === Orane Steadwife === <gallery> Image:Orane-small.jpg </gallery> === Pelaskos the Fisher === <gallery> Image:Pelaskos1-small.jpg Image:Pelaskos2-small.jpg </gallery> [[Category:Glorantha]] 828f6488361362ee243097c389b72b8ec2c7eec7 0 1500 1649 2008-02-03T11:17:08Z Neil 2 wikitext text/x-wiki <blockquote>by Terry Harvey-Chadwick, based on official material from [http://www.glorantha.com Issaries Inc.]</blockquote> {| cellpadding=3 align=right width=50% border=1 | === Clan Statistics === * '''Lunar Friendship Rating:''' -1 * '''Wealth:''' 5W2 * '''Keywords:''' ''Women:'' +4 Cooking; ''Men:'' +3 Initiate or Devotee of Orlanth; ''All:'' Skating +4 * '''Attitudes:''' Fear Chaos 17, Fear Dragons 15, Friendly to Volsaxings 17, Hate Arkat 1W, Hate Lunars 14, Hate Trolls 3W * '''Clan Secret:''' Hit Jagrekriand with Missile (+10 to Ranged Combat skill once per contest, when appropriate) * '''Motto:''' "Violence is always an option" |} The Tertorae clan are part of the Cinsina tribe, also known as the Brave Tribe, but called the Wolfslayers amongst themselves. Their lands encompass the area enclosed by the Solfint and Lorthing Rivers to the north and west, Bragi's Ford and the High Hills to the south, and part of the lands between the Vinga's Throw Hills and the Bull Hills to the East. __TOC__ The Tertorae are a traditional balanced Orlanthi farming, hunting and herding clan, although a large minority worship Elmal. The main stead is Wulfsstead in a deep valley of the Thunderstone Heights, as it is well protected and central to most of the major places of importance. Other important steads include Founders Stead, Broken Fang Stead, Lightstead, Upper Heights Stead, and Harststead. The clan consists of approximately 1400 adults, making it the most numerous clan of the Cinsina. == Political Affiliations == The Tertorae tula holds the largest Cinsina Ernalda temple, a small Vinga temple, and a shrine to Helamakt. The Tertorae are allied with the Decamoli and Drutorae clans. Relationships with the Halfari clan are strained as Ivar Quickstep, chieftain of the Halfari, also claims the Tribal Kingship. Tertorae warriors often raid into the Greenhaft, Barlamani and Marthiording clans of the Culbrea tribe. They have also recently started raiding the Gwandor. Raiding of the Blueberry and Gorde clans stopped when they were forced into the Cinsina tribe by the Lunars. Relations with Wulfsland are cordial. The current clan Chieftain and Tribal King is Ivartha the Skinner, a Vingan chosen to be politically neutral to the Lunars. == The Tertorae Clan Ring == === Lightbringer Ring === ;Orlanth :Ivartha the Skinner (Vinga & Orendana) ;Issaries :Jonrik Goodvoice (Harst) ;Lhankor Mhy :Enfrew Sharptongue (Andrin) ;Chalanna Arroy :Sora Gentletouch (Jera) ;Eurmal :Orlkarl Crookleg ;Flesh Man :Hengall Venharlson (Durev) ;Elmal :Ranulf the Whisperer (Beren) === Ginna Jar === Vorana Ericsdottir (Harst), Huma Lightbreeze (Ormalaya), Bavari Torkilson (Elmal), Ketil the Axe (Orlanthcarl), Arkell Brightspear (Elmal), Bragi Stonehead (Voriof), Kort Thundersling (Hedkoranth), Yrsa the Shrill (Vela Matchmaker), Erica Longhair (Orane Steadwife), Torkil the Stout (Poverri), Astrid Honeydew (Minlinster), Olaf the Lofty (Ohorlanth), Angantir Longarm (Harst), Janerra Keeneye (Manhome), Kestra Fleetfoot (Vinga), Meera Cowsong (Uralda), Morag Blue-eye (Esra) === Other Important People === ;Clan Champion :Pallashee LongAxe (Kargan) ;Warband Leader :Halgrim Sharpsword (Starkval) ;Master Hunter :Korst the Tracker (Odayla) == Joral Trollkiller: The Clan Wyter == {| cellpadding=3 align=right width=50% border=1 | === Wyter Statistics === * '''Physical Manifestation:''' Ancient Bronze Spear * '''Communication Manifestation:''' All spearheads hum. * '''Awareness''' 3W3: Sense Trolls; Sense False Friends * '''Defense''' 17W2: Ignore Restraints; Defend Against Trolls * '''Blessings''' 19W2: Sacrifice Dance +5; Thunder Blessing; Blow Down Foe; Skate Marvels; Dwarf Friend |} Joral was a warrior of Helamakt during the Vingkotling age. During that time his clan was beset by trolls, who ate the crops, the wild woods, the cattle and other domestic animals, and many of our ancestors. Joral defended his tula with the powers of the wind and rain, blowing down their ranks and slaying them with his spear. The trolls feigned peace, captured him and bound him in lead chains. Joral escaped and fled over the ice on skates made from the shinbones of the troll guards who tried to stop him. Joral was then captured by dwarves, but convinced them of his friendship, and they aided him against the trolls. In the Greater Darkness, Joral's clan gave hospitality to many different people, outcasts and refugees. Several times a group of guests planned to betray their hosts, but Joral discovered their plotting and showed them for what they truly were. Joral was slain by the great troll champion Gork, but Joral's spear was used by another warrior to avenge him. Joral agreed to become the clan Wyter and look after his descendants forever more after his death. Since then he has protected us well, even warning us when the Lunars came with the false promises, so we do not trust them. [[Category:Tertorae]] cdfdbbc40020c7aab2b2d3f84014043d465eeb6b 1652 1649 2008-02-03T11:26:55Z Neil 2 wikitext text/x-wiki <blockquote>by Terry Harvey-Chadwick, based on official material from [http://www.glorantha.com Issaries Inc.]</blockquote> {| cellpadding=3 align=right width=50% border=1 | === Clan Statistics === * '''Lunar Friendship Rating:''' -1 * '''Wealth:''' 5W2 * '''Keywords:''' ''Women:'' +4 Cooking; ''Men:'' +3 Initiate or Devotee of Orlanth; ''All:'' Skating +4 * '''Attitudes:''' Fear Chaos 17, Fear Dragons 15, Friendly to Volsaxings 17, Hate Arkat 1W, Hate Lunars 14, Hate Trolls 3W * '''Clan Secret:''' Hit Jagrekriand with Missile (+10 to Ranged Combat skill once per contest, when appropriate) * '''Motto:''' "Violence is always an option" |} The Tertorae clan are part of the Cinsina tribe, also known as the Brave Tribe, but called the Wolfslayers amongst themselves. Their lands encompass the area enclosed by the Solfint and Lorthing Rivers to the north and west, Bragi's Ford and the High Hills to the south, and part of the lands between the Vinga's Throw Hills and the Bull Hills to the East. __TOC__ The Tertorae are a traditional balanced Orlanthi farming, hunting and herding clan, although a large minority worship Elmal. [[Image:Tertorae-tula-small.jpg|left|thumb|200px|Tertorae clan tula]] The main stead is Wulfsstead in a deep valley of the Thunderstone Heights, as it is well protected and central to most of the major places of importance. Other important steads include Founders Stead, Broken Fang Stead, Lightstead, Upper Heights Stead, and Harststead. The clan consists of approximately 1400 adults, making it the most numerous clan of the Cinsina. == Political Affiliations == The Tertorae tula holds the largest Cinsina Ernalda temple, a small Vinga temple, and a shrine to Helamakt. The Tertorae are allied with the Decamoli and Drutorae clans. Relationships with the Halfari clan are strained as Ivar Quickstep, chieftain of the Halfari, also claims the Tribal Kingship. Tertorae warriors often raid into the Greenhaft, Barlamani and Marthiording clans of the Culbrea tribe. They have also recently started raiding the Gwandor. Raiding of the Blueberry and Gorde clans stopped when they were forced into the Cinsina tribe by the Lunars. Relations with Wulfsland are cordial. The current clan Chieftain and Tribal King is Ivartha the Skinner, a Vingan chosen to be politically neutral to the Lunars. == The Tertorae Clan Ring == === Lightbringer Ring === ;Orlanth :Ivartha the Skinner (Vinga & Orendana) ;Issaries :Jonrik Goodvoice (Harst) ;Lhankor Mhy :Enfrew Sharptongue (Andrin) ;Chalanna Arroy :Sora Gentletouch (Jera) ;Eurmal :Orlkarl Crookleg ;Flesh Man :Hengall Venharlson (Durev) ;Elmal :Ranulf the Whisperer (Beren) === Ginna Jar === Vorana Ericsdottir (Harst), Huma Lightbreeze (Ormalaya), Bavari Torkilson (Elmal), Ketil the Axe (Orlanthcarl), Arkell Brightspear (Elmal), Bragi Stonehead (Voriof), Kort Thundersling (Hedkoranth), Yrsa the Shrill (Vela Matchmaker), Erica Longhair (Orane Steadwife), Torkil the Stout (Poverri), Astrid Honeydew (Minlinster), Olaf the Lofty (Ohorlanth), Angantir Longarm (Harst), Janerra Keeneye (Manhome), Kestra Fleetfoot (Vinga), Meera Cowsong (Uralda), Morag Blue-eye (Esra) === Other Important People === ;Clan Champion :Pallashee LongAxe (Kargan) ;Warband Leader :Halgrim Sharpsword (Starkval) ;Master Hunter :Korst the Tracker (Odayla) == Joral Trollkiller: The Clan Wyter == {| cellpadding=3 align=right width=50% border=1 | === Wyter Statistics === * '''Physical Manifestation:''' Ancient Bronze Spear * '''Communication Manifestation:''' All spearheads hum. * '''Awareness''' 3W3: Sense Trolls; Sense False Friends * '''Defense''' 17W2: Ignore Restraints; Defend Against Trolls * '''Blessings''' 19W2: Sacrifice Dance +5; Thunder Blessing; Blow Down Foe; Skate Marvels; Dwarf Friend |} Joral was a warrior of Helamakt during the Vingkotling age. During that time his clan was beset by trolls, who ate the crops, the wild woods, the cattle and other domestic animals, and many of our ancestors. Joral defended his tula with the powers of the wind and rain, blowing down their ranks and slaying them with his spear. The trolls feigned peace, captured him and bound him in lead chains. Joral escaped and fled over the ice on skates made from the shinbones of the troll guards who tried to stop him. Joral was then captured by dwarves, but convinced them of his friendship, and they aided him against the trolls. In the Greater Darkness, Joral's clan gave hospitality to many different people, outcasts and refugees. Several times a group of guests planned to betray their hosts, but Joral discovered their plotting and showed them for what they truly were. Joral was slain by the great troll champion Gork, but Joral's spear was used by another warrior to avenge him. Joral agreed to become the clan Wyter and look after his descendants forever more after his death. Since then he has protected us well, even warning us when the Lunars came with the false promises, so we do not trust them. [[Category:Tertorae]] daf03fe03e2d68369c2eb10500ad37093f10d49e 14 1501 1650 2008-02-03T11:21:28Z Neil 2 wikitext text/x-wiki Here you will find details of the Tertorae clan of the Cinsina tribe of Northern Sartar. A [http://www.glorantha.com/new/images/MapDragonPass.jpg map of Sartar] can be found on the [http://www.glorantha.com Issaries, Inc. website]. [http://www.celtic-webs.com/glorantha/index.html Wesley Quadros's site] has [http://www.celtic-webs.com/glorantha/middle/maps/maps.html lots of maps], including Northern Sartar, where the Cinsina lands are. [[Category:Glorantha]] 4dad414d820beb7ef2c7fb89fcec18646ed83b25 6 1502 1651 2008-02-03T11:23:22Z Neil 2 A map of the [[Tertorae clan]] tula. wikitext text/x-wiki A map of the [[Tertorae clan]] tula. 72fe7a8fb925e065f6933376bde4d3790a2d666c 0 1503 1653 2008-02-03T11:35:04Z Neil 2 wikitext text/x-wiki <blockquote>by Terry Harvey-Chadwick</blockquote> == Asrelia's Age, The God Time == In Asrelia's Age gods and people walked the world together, and neither could tell which was which. Our ancestor was the First Ancestor, called Grandfather Life. With Grandmother Life he had hundreds of children who, in turn, had hundreds of their own. The cosmos was at peace, but slowly awareness woke. Our ancestor's awareness awoke when she realised that she was different from other people, for she was among the Treasured People. Our ancestor lived in the beautiful, luxurious vaults and rooms of Asrelia. She grew up with the Rich Swan, one of the Earth Treasures of Asrelia. === The First Ceremony === At first no-one was ever hungry, for Asrelia contained enough food for everyone. But suddenly, some people were! Ernalda stopped the hunger by teaching people how to do the right ceremonies. These allowed people to find her sister, Esrola, who had been trapped behind a terrible mask by Angdertha. This was the first worship, one of the actions that separated mortals from deities. Our ancestor led the Sacrifice Dance, which our women still perform every year in the spring and discovered the secrets of preparing the bounty of Ernalda. == The Age of Stagnation == The Bright Emperor enslaved everyone and demanded absolute obedience that was little more than death-of-the-soul. But there was once force he could not control, and that was freedom. One little freedom became many, and they began to bring bigger freedoms. Finally, the great sky lay upon the fertile earth and begot Umath Ever-changing, the First Storm, who would bring change to the Bright Emporers stagnant realm. === The Umath Roots === Umath was an upstart god at first, with only a few crazy and dangerous brothers at his side. Gradually he gathered gods and men to his band and, with their help, reshaped the world. Our ancestors were one of the first mortals to help Umath when he separated the sky and earth, and was instrumental in this first act of violence. == The Storm Tribe Age == Umath's youngest and wisest son, Orlanth, made the world as it is today. He married Ernalda, formed the Storm Tribe, and established the first laws and customs that made us what we are today. Orlanth is the King of the gods, whom men follow and women desire, and around whom the winds flock. Where the Bright Emperor sought to control people through fear, force and hollow laws, Orlanth gathered his people through force of virtue, character and example. Our ancestors were amongst the first to recognise Orlanth's greatness and followed him from the start. === The Wedding of Orlanth and Ernalda === Orlanth wooed and wed Ernalda. They invited the whole world to attend their nuptials and, since our ancestors were amongst their friends, they attended. When they entered the sacred space, they were asked if they wanted to stand on the right (Orlanth) or the left (Ernalda) side. Our ancestors chose to guard the sacred circle, with Elmal. === The First Ancestor Clans === At Ernaldas urging Orlanth made the Storm Tribe. He recognised the First Ancestor Clans. Grandfather Life was killed, so all of his descendants are doomed to die as well. The First Ancestor Clans recognise the ultimate distinction between gods and humans. Our ancestors joined the Urlanthing clan, the people of Urlanth, as primal member of the Storm Peoples. == The Vingkotling Age == Vingkot was the first human to rule over many clans. The Vingkotlings were great, with full baskets and fat herds, and always victorious over their foes. The people set up many altars, where the gods lived, and so established the relations of worship. King Vingkot led and sent many expeditions against the enemies who wanted to take it away. Vingkot is still worshipped for the great deeds he did, and for his leadership of the Storm Tribe when Orlanth went away. === Ancient Enemies === The foes of the Vingkotlings were many. As time went on we found ourselves fighting the same foes over and over again. One enemy in particular, the Darkness People, or Trolls, plagued our people consistently during this era, eating everything on our lands that they came across. Our ancestors fought these enemies so often we developed special magic to combat them. Even today our clan does not get on very well with the descendants of our Ancient Enemy, and our Wyter is especially effective against them. === The Gods War === The Gods War was long and destructive. When the gods clashed mountain ranges fell, the sky broke, and the earth was torn to shreds, like a rabbit in the jaws of a wolf. Our ancestors fought a lot, for those who did not fight died, and have no descendants today. Legendary heroes fought in those ages. They are many, although some are unknown outside our clan, who tell their tales. Our clan had a friendship with Helamakt, the Great Defender, helping him at the Battle of Ur Ain, where he drove the salty gods of the sea onwards like froth in a storm. Due to this ancient friendship we now maintain a small temple to Helamakt in the High Hills, near the Upper Heights Stead. Our clan wyter is able to draw upon the fighting storm and blow our foes over, making them easier to defeat. === The Three Red Peak Battle === One time among many, Vingkot travelled north to fight Dara Happa. Although he had done this before, this time he only took a few warriors with him, because he wanted to travel secretly. He took the three sons of Orand, and it was good that he did, because each if them saved the warband when they were in danger of failure. Our clan is one of those few whose ancestors participated in this great raid. Umak Redshot was a son of the warrior Orand, who rode in the warband of Kodig Vingkotsson. Umak made a special sacrifice to Orlanth, as shown by the scars cut in all the men of our clan. He used this power when Jagrekriand attacked. He had time for one shot, and it struck the enemy god hard. All of our warriors are able to learn this feat. === Subject Peoples === When the world fell apart, Vingkot welcomed refugees and reorganised the devastated clans, so that all might survive. He asked us to shelter the people of a ruined clan but let us choose what status to give them. We took in the Korsto Viv, the People of the Rivers, and gave them equal rights: the farmers became carls and the warriors became weapon thanes. Because of this, our clan does not agree with the taking of thralls. == The Chaos Age == Eventually, the Great Darkness descended on Glorantha. Many gods died, others slept, and even Orlanth abandoned his people to try to save the world. Everything in the universe seemed hostile, and was at least uncaring. At last, only monsters, demons, formless Chaos, and their victims remained. Many were so terrible that nothing could withstand them. Clans, tribes, kingdoms and nations disappeared in this terrible time. Some things could be resisted with fire, magic and spear, and all people alive today are descended from someone who survived this time. During this time many beings came to us, feigning friendship, only to betray us, so many times that our Wyter has a sensitivity to feigned friendships, and is able to warn us when these false friends arrive. Everyone who survived the Great Darkness did so because of a survival secret they learned. Most were saved by a living god, who protected them until Orlanth and Ernalda returned. Our clan survived by always moving on when danger threatened, by skating away on the ice. Since then all people in our clan have been good at skating on ice. === I Fought We Won === The entire world was on the brink of destruction and it came down to one person to resist Wakboth, the Devil. That person did, and so the world was saved. You learned in the initiation rites of our clan that you were that individual. Heort is the one who teaches you this, and so you are called a Heortling. Heort reorganised the people onto new clans. Many people came together. Although they became one people they had different traditions and ways of governing themselves. Our clan decided to emulate Orlanth and formed a Storm Ring, first established when Orlanth gave out the great God Treasures to his followers. == The Silver Age == King Heort guided our people out of the Great Darkness. He re-instituted sacrifice to the gods, organised people into clans, gave out the new laws for society, and arranged the first new tribe. In his honour we became Heortlings, part of the Koroltes Tribe, which no longer exists. He also arranged alliances with all the nearby Elder Races and formed the Unity Council. The Unity Council of Silver Age Dragon Pass was a unique and extraordinary event. Many ancient feuds and hostilities were set aisde for the sake of harmony. The unity allowed the peoples to cast off Chaos and to thrive in comparison with the rest of the miserable Darkness-plagued world. For the first time ever humans and the Elder Races worked co-operatively. During this time of peace and co-operation our ancestors made alliances with the Mostali, whom we had previously called dwarves. The Mostali taught us the proper gestures and words to make a Mostali greeting, so that they will meet with us if it seems profitable or harmless. == The Hero Age: History == The unity between peoples established during the Silver Age did not last forever. Terrible things happened, and great wars destroyed whole lands and races. Some peoples remembered the old ways to live in the new world, and so they survived. === Lokamayadon === Orlanth always encouraged personal ambition. No-one thought there could be too much personal advancement until a priest called Lokamayadon took upon himself the powers of High Storm. He tried to replace Orlanth, peacefully at first and then by killing all who worshipped God. Most people went along with him because he was powerful, but most Heortlings resisted him, although it cost them greatly. Our ancestors fought the High Storm people when it was safe to do so, and the clan wouldn't be endangered too much. All Heortlings hate Lokamayadon and what he did, and when Lokamayadon's name is mentioned our clan automatically makes the sign of protection from evil gods with our right hands. === Arkat === Lokomo helped create a new god called Gbaji, which brought Chaos back. It would have destroyed the world except that the great hero Harmast Barefoot performed the Lightbringers Quest and brought back the destroyer of Gbaji. Arkat was the destroyer. He was a man who had lived before, a sorcerous demon in the form of a man, or maybe just a trickster. Arkat helped everybody, then betrayed them. He deserted his army when it fought Dara Happa. He left behind Durega, his lover; and he finally even left Harmast in the clutches of the She Shadows, his worst foe, so that he could betray the whole human race and become a troll. Everyone hates Arkat, even though he destroyed Gbaji and saved the world. Our clan suffered terribly at his hands, but we survived, even though we sent all of our warriors to fight with him. === The Empire of Wyrm's Friends === During the Imperial Age we were approached by a dragon, in the shape of a human, but with the soul of a dragon. The dragons had always been hostile or, at the best, utterly uncaring, but now it offered to share knowledge with us. Orlanth has always been the dragon-slayer, but many priests of other clans embraced these new ways and formed the EWF. Our ancestors resisted the dragons, for we would never believe them, although it cost us dear. Since that time we have always had reason to fear dragons and their kin. === The Dragonkill War === The EWF was corrupted by dragon powers and finally destroyed when the Dragonewts destroyed its leaders. Afterwards, several great armies invaded dragon Pass from the north, east and south. After decades of warfare, hundreds of dragons (including the great dragons) appeared, some as big as the sky, and ate every human being they found, hostile or otherwise. A few incredibly lucky people survived, said to number only en humans. Any hatred people felt for the dragons was replaced by unconquerable fear. We learnt that knowledge isn't harmful in itself, only the way it is used. After the Dragonkill a series of Death runes were carved in stone and erected as the Cross Line, beyond which lay a land forever cursed. "Beyond that line is dragon, this side is us. None may cross except to make pain, horror and terrible death." For centuries, no-one even went close to it. Anyone who dared to cross was never heard from again. == Historical Memory == After the Dragonkill our ancestors lived in Volsaxar (Northern Heortland). We were a part of the Volsaxi Tribe, whose kings had no dynasties and operated in the old way, as rings. The clans were independent, always feuding with the Trolls of the Shadow Plateau and Kitori Wilds. Whitewall is a famous stronghold of Volsaxar, and Smithstone is the ancient capital. However, our kings were themselves vassals of Ezkankekko, a troll demigod who lived in a huge volcanic palace on the Shadow Plateau. Ezkankekko ruled with justice and indifference, and generally kept the peace. === The Resettlement === In the year 1313ST a man swan ashore and began to cause trouble in the land of Kethaela. He confronted the Only Old One, Ezkankekko, and killed him. Thereafter, the usurper called himself The Pharaoh. The Pharaoh went throughout Kethaela and established his rules over the native peoples. When he came to our kingdom he used deceitful magic to overthrow our king, and Heortland became part of his kingdom and civil war broke out. At the same time as this was happening we heard that the lands to the north, in forbidden Dragon Pass, were once more open to humans. We decided to explore these new lands, so we planned and, when we were properly equipped and prepared we found some good new land and settled in Dragon Pass. As clans settled they began to fight one another. Some were conquered, some received tribute from dozens of others, and some formed alliances. We joined early into the Cinsina tribe and gained prestige in the tribe. === King Sartar === About 150 years ago a man named Sartar came to our land of warring tribes. His ways were new and strange, but his magic was powerful. He made this place into the kingdom that now bears his name. He gained most admiration from us when he immolated himself and rose into the sky, leaving behind The Flame of Sartar, which continued burning without fuel until the Lunars extinguished it. === The Lunar War === High King Sartar exchanged ambassadors with the king of Tarsh to promote peace and trade. Men and women in red robes came after his apotheosis, and for the first time people in Sartar learned about the Lunar goddess. The Red Moon had risen in 1247, a bad omen that surprised and frightened the Heortlings. Everyone became uncomfortable when they learned that the missionaries worshipped the red thing in the sky, for all the Sartarite prophesies warned that the Red Moon was a foe. Nonetheless, the Red Goddess' missionaries came through Sartar, seeking converts to her way. We gave the missionaries hospitality at this time. However, the Lunar Empire conquered Tarsh many years ago , then they tried to conquer Sartar as well. The great High King Tarkalor led everyone to many years of victory in battles with the Lunar Empire, and we fought honourably, but not insanely, as we also had the overall welfare of the clan to think of. Then there was the terrible defeat at Grizzly Peak, and the Lunars won the war. They occupied Sartar and closed down all the city temples of Orlanth, levied taxes, and enslaved those they deemed criminals or rebels. They imposed many laws that were impossible to obey, just so they had excuses to tax the clans and arrest anyone. We chose to bide our time until a good day for rebellion came. With no Orlanthi king on the throne of Sartar, many of our countrymen took advantage of the opportunity to settle old grudges or steal from neighbours. The tribes fell to fighting one another. However, along with some other clans, we attacked the Telmori, a werewolf race that eats people and has never worshipped our gods. Cattle were given and taxes lifted to all those who gave aid. Last year many clan chieftains and tribal kings, led by the Vingan warrior Kallyr Starbrow, took part in an uprising against the Lunars. We said that it was not the time for rebellion, but some of our warriors fought, even though we forbade them to. When the Lunars conquered Sartar they sent assassins, demons and gods to kill all the heirs of the house of Sartar. However, during Starbrows Rebellion an heir was found in the south, a scholar named Temertain. He is now prepared to be the legendary "Philosopher King". His dynastic blood has been proven, but he could not light the flame of Sartar that the Lunars extinguished in 1602. So far, his philosophy seems to be close collaboration with the Lunars. However, even Lunar supporters know that he is weak, just a Lunar puppet. We hate him, but there is nothing to be done. The Lunars have made the Durulz scapegoats for the Kallyr's rebellion, and put a bounty on their heads. However, due to our ancient friendship with the river, we have never killed a Durulz just to make our lives a bit easier. == Recent Events == === 1607=== The neighbouring Maboder tribe are wiped out by the Telmori. Cinsina warriors join Duke Jomes Wulf in conquering Telmori. The Maboder lands are given to Jomes Wulf and renamed Wulfsland. === 1613: Starbrow Rebellion === The Dinacoli tribe attack Dangerford but are beaten off by Cinsina warriors. The Lunars march on Dangerford and the Decamoli surrender with good terms. Dangerford is now occupied by Lunar garrison, although the Decamoli still live there and collect tolls from river traffic. The Blueberry and Gorde clans are forcibly reassigned to the Cinsina tribe from the Culbrea by the Lunars, after Starbrow's Rebellion is crushed. [[Category:Tertorae]] 25937e91a6d4eaad4d5d885a3a65b166ec85e4da 0 1504 1654 2008-02-03T11:50:19Z Neil 2 wikitext text/x-wiki <blockquote>by Terry Harvey-Chadwick</blockquote> == Bloodlines == [[Image:Tertorae-tula-small.jpg|left|thumb|200px|Tertorae clan tula]] There are four bloodlines in the Tertorae: ;Emanari :Descended from Emanara, a daughter of Queen Cinsina. By far the most numerous bloodline, including the current Tribal King, Ivartha the Skinner, amongst its members. ;Korting :Descended from Kort the Elder, a priest of Ohorlanth the Great Storm, and a prolific heroquestor. ;Harivari :Descended from Harivig the Red, a weaponthane of Queen Cinsina. This bloodline tends towards Elmal worship, and breed fine horses. ;Bragii :Descended from Bragi Closemouth, a thane of Emanara. == Steads == {| cellpadding=3 ! Stead !! Bloodline(s) !! Adults !! Notes !! Household Head |- | Founders Stead || Emanari || 74 || Oldest stead. Market here. || Vorana Ericsdottir |- | Wulfsstead || Korting / Emanari || 80/21 || Chieftain's Stead || Enfrew Sharptongue* |- | Lightstead || Harivari || 87 || Mainly worship Elmal. Breed horses || Ranulf the Whisperer* |- | Hunters Lodge || None || - || Used by hunters. No permanent residents || Huma Lightbreeze |- | Woods Stead || Harivari || 58 || || Bavari Torkilson |- | Ketilsstead (in woods) || Harivari || 47 || || Ketil the Axe |- | Ormson Stead || Harivari || 62 || || Arkell Brightspear |- | Vingasstead || Emanari || 70 || Produced many redheads || Ivartha the Skinner* |- | Lost Sheep Stead|| Emanari || 77 || Herd Sheep || Bragi Stonehead |- | Kortstead || Korting || 73 || Founded by Kort || Kort Thundersling |- | Varnisstead ||`Korting || 62 || || Yrsa the Shrill |- | Crown Stead || Emanari || 91 || || Erica Longhair |- | Fishers Stead || Bragii || 53 || Mainly fishermen || Torkil the Stout |- | Ketilstead || Korting / Emanari || 39/48 || Known for their mead || Astrid Honeydew |- | Harststead || Emanari || 81 || || Jonrik Goodvoice* |- | Upper Heights Stead || Korting || 43 || || Olaf the Lofty |- | Bragistead || Bragii || 69 || Founded by Bragi || Hengall Venharlson* |- | Broken Fang Stead || Emanari || 77 || || Angantir Longarm |- | Deer Stead || Bragii || 62 || || Janerra Keeneye |- | Ernalda's Temple || Emanari || 79 || Main Cinsina temple || Sora Gentletouch* |- | Vinga Temple || Emanari || 39 || Founded by Emanara || Kestra Fleetfoot |} * Member of the Clan Ring 0b3ff4f354acc60f02129d8621b1c3d35b57baf1 1655 1654 2008-02-03T11:50:50Z Neil 2 wikitext text/x-wiki <blockquote>by Terry Harvey-Chadwick</blockquote> == Bloodlines == [[Image:Tertorae-tula-small.jpg|left|thumb|200px|Tertorae clan tula]] There are four bloodlines in the Tertorae: ;Emanari :Descended from Emanara, a daughter of Queen Cinsina. By far the most numerous bloodline, including the current Tribal King, Ivartha the Skinner, amongst its members. ;Korting :Descended from Kort the Elder, a priest of Ohorlanth the Great Storm, and a prolific heroquestor. ;Harivari :Descended from Harivig the Red, a weaponthane of Queen Cinsina. This bloodline tends towards Elmal worship, and breed fine horses. ;Bragii :Descended from Bragi Closemouth, a thane of Emanara. == Steads == {| cellpadding=3 ! Stead !! Bloodline(s) !! Adults !! Notes !! Household Head |- | Founders Stead || Emanari || 74 || Oldest stead. Market here. || Vorana Ericsdottir |- | Wulfsstead || Korting / Emanari || 80/21 || Chieftain's Stead || Enfrew Sharptongue* |- | Lightstead || Harivari || 87 || Mainly worship Elmal. Breed horses || Ranulf the Whisperer* |- | Hunters Lodge || None || - || Used by hunters. No permanent residents || Huma Lightbreeze |- | Woods Stead || Harivari || 58 || || Bavari Torkilson |- | Ketilsstead (in woods) || Harivari || 47 || || Ketil the Axe |- | Ormson Stead || Harivari || 62 || || Arkell Brightspear |- | Vingasstead || Emanari || 70 || Produced many redheads || Ivartha the Skinner* |- | Lost Sheep Stead|| Emanari || 77 || Herd Sheep || Bragi Stonehead |- | Kortstead || Korting || 73 || Founded by Kort || Kort Thundersling |- | Varnisstead ||`Korting || 62 || || Yrsa the Shrill |- | Crown Stead || Emanari || 91 || || Erica Longhair |- | Fishers Stead || Bragii || 53 || Mainly fishermen || Torkil the Stout |- | Ketilstead || Korting / Emanari || 39/48 || Known for their mead || Astrid Honeydew |- | Harststead || Emanari || 81 || || Jonrik Goodvoice* |- | Upper Heights Stead || Korting || 43 || || Olaf the Lofty |- | Bragistead || Bragii || 69 || Founded by Bragi || Hengall Venharlson* |- | Broken Fang Stead || Emanari || 77 || || Angantir Longarm |- | Deer Stead || Bragii || 62 || || Janerra Keeneye |- | Ernalda's Temple || Emanari || 79 || Main Cinsina temple || Sora Gentletouch* |- | Vinga Temple || Emanari || 39 || Founded by Emanara || Kestra Fleetfoot |} * Member of the Clan Ring [[Category:Tertorae]] bd9d6a481010dba1af5bc60ae65ddd77c24e9b1a 14 1458 1657 1585 2008-02-03T11:57:02Z Neil 2 wikitext text/x-wiki Welcome to my my role-playing pages. Historically, much of my roleplaying is spent in and around [[:Category:Glorantha|Glorantha]] and similarly, the majority of the material in these pages relates to that. All of my role-playing activity is done at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games Club]. We meet on Tuesday evenings. New members are always welcome. 750782e98ca30460be43ef72341af9bf952a29e0 0 1505 1659 2008-02-03T12:39:02Z Neil 2 wikitext text/x-wiki == Contest summary == For '''Simple''' and ''Extended'' contests {| cellpadding=3 border=1 ! !! Critical !! Success !! Failure !! Fumble |- ! Critical | '''Low = Marginal'''<br>''High = Transfer &frac12;&times;'' | '''Minor'''<br>''Transfer 1&times;'' | '''Major'''<br>''Transfer 2&times;'' | '''Complete'''<br>''Transfer 3&times;'' |- ! Success | '''Minor'''<br>''Transfer 1&times;'' | '''Low = Marginal'''<br>''High = Forfeit &frac12;&times;'' | '''Minor'''<br>''Forfeit 1&times;'' | '''Major'''<br>''Forfeit 2&times;'' |- ! Failure | '''Major'''<br>''Transfer 2&times;'' | '''Minor'''<br>''Forfeit 1&times;'' | '''Low = Marginal'''<br>''High = Forfeit &frac12;&times;'' | '''Minor'''<br>''Winner forfeit 1&times;'' |- ! Fumble | '''Complete'''<br>''Transfer 3&times;'' | '''Major'''<br>''Forfeit 2&times;'' | '''Minor'''<br>''Forfeit 1&times;'' | '''Tie (both marginal defeat)'''<br>''Tie (both forfeit &frac12;&times;)'' |} == Outcomes == {| cellpadding=3 border=1 ! Outcome !! AP Equivalent !! Consequences !! Combat Outcome !! Augmentation !! AP Loan |- ! Complete Victory | || || || Double Bonus || Target gains |- ! Major Victory | || || || Bonus || Target gains<br>Lender loses |- ! Minor Victory | || || || Bonus || Target gains<br>Lender loses |- ! Marginal Victory | || || || &frac12; Bonus || Target gains<br>Lender loses |- ! Tie | || || || No effect || No effect |- ! Marginal Defeat | 0 to -10 || None significant || Dazed || &frac12; Penalty || Lender loses &frac12; |- ! Minor Defeat | -11 to -20 || -1/10 victor's ability || Hurt || &frac12; Penalty || Lender loses |- ! Major Defeat | -21 to -30 || -50% || Injured || Penalty || Both lose |- ! Complete Defeat | -31 or more || No actions || Dying || Penalty || Both lose |} == Augmentation == For variable augmentations, Resistance is Desired bonus &times; 5. For automatic augmentations, bonus is: {| ! Ability !! Augment |- | 5 - 14 || +1 |- | 15 - 4W || +2 |- | 5W - 14W || +3 |- | 15W - 4W2 || +4 |- | 5W2 - 14W2 || +5 |} == AP Loan == Resistance is number of APs loaned A [http://www.glorantha.com/support/GameAids.pdf synopsis of the HeroQuest] rules can be found on the [http://www.glorantha.com/support/ support page] of the [http://www.glorantha.com official Glorantha website]. <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] 7044fb9565918445b6bd2e2fc193d221f4df2b36 1662 1659 2008-02-08T14:19:17Z Neil 2 wikitext text/x-wiki == Contest summary == For '''Simple''' and ''Extended'' contests {| cellpadding=3 border=1 ! !! Critical !! Success !! Failure !! Fumble |- ! Critical | '''Low = Marginal'''<br>''High = Transfer &frac12;&times;'' | '''Minor'''<br>''Transfer 1&times;'' | '''Major'''<br>''Transfer 2&times;'' | '''Complete'''<br>''Transfer 3&times;'' |- ! Success | '''Minor'''<br>''Transfer 1&times;'' | '''Low = Marginal'''<br>''High = Forfeit &frac12;&times;'' | '''Minor'''<br>''Forfeit 1&times;'' | '''Major'''<br>''Forfeit 2&times;'' |- ! Failure | '''Major'''<br>''Transfer 2&times;'' | '''Minor'''<br>''Forfeit 1&times;'' | '''Low = Marginal'''<br>''High = Forfeit &frac12;&times;'' | '''Minor'''<br>''Winner forfeit 1&times;'' |- ! Fumble | '''Complete'''<br>''Transfer 3&times;'' | '''Major'''<br>''Forfeit 2&times;'' | '''Minor'''<br>''Forfeit 1&times;'' | '''Tie (both marginal defeat)'''<br>''Tie (both forfeit &frac12;&times;)'' |} == Outcomes == {| cellpadding=3 border=1 ! Outcome !! AP Equivalent !! Consequences !! Combat Outcome !! Augmentation !! AP Loan |- ! Complete Victory | || || || Double Bonus || Target gains |- ! Major Victory | || || || Bonus || Target gains<br>Lender loses |- ! Minor Victory | || || || Bonus || Target gains<br>Lender loses |- ! Marginal Victory | || || || &frac12; Bonus || Target gains<br>Lender loses |- ! Tie | || || || No effect || No effect |- ! Marginal Defeat | 0 to -10 || None significant || Dazed || &frac12; Penalty || Lender loses &frac12; |- ! Minor Defeat | -11 to -20 || -1/10 victor's ability || Hurt || &frac12; Penalty || Lender loses |- ! Major Defeat | -21 to -30 || -50% || Injured || Penalty || Both lose |- ! Complete Defeat | -31 or more || No actions || Dying || Penalty || Both lose |} == Augmentation == For variable augmentations, Resistance is Desired bonus &times; 5. For automatic augmentations, bonus is: {| ! Ability !! Augment |- | 5 - 14 || +1 |- | 15 - 4W || +2 |- | 5W - 14W || +3 |- | 15W - 4W2 || +4 |- | 5W2 - 14W2 || +5 |} == AP Loan == Resistance is number of APs loaned A [http://www.glorantha.com/support/GameAids.pdf synopsis of the HeroQuest] rules can be found on the [http://www.glorantha.com/support/ support page] of the [http://www.glorantha.com official Glorantha website]. <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] [[Category:HeroQuest]] 71b5302ca2313812e23a15ac9a63e5806a6bbbdf 0 1506 1660 2008-02-03T12:43:14Z Neil 2 wikitext text/x-wiki Here are some pearls of wisdom I've learnt from narrating a HeroQuest campaign. In essence, my tips are: __TOC__ == Relax == It's very easy to get involved in debates over 'The One True Way' to play HeroQuest. Don't. Remember, HeroQuest is just a game, and a HeroQuest session is about you and a bunch of mates getting together to have fun. Don't worry too much about whether you're doing things according to the letter of the rulebook, or whether some obscure offering from Greg's wastebasket contrdicts what someone's just said. If people are having fun, it's a good game. == Learn to say 'Yes' == HeroQuest is a game of improvisation. There are no defined skill lists, feat lists, or whatever. PCs will improvise their abilities as they go along. You should allow, nay, encourage this. It can take some getting used to (it did for me), particularly if you're used to the more traditional games with fully defined abilities. Practice saying 'yes.' Next time a player asks you if a PC can use ability X to do Y, say 'yes' first, then worry about whether it makes sense. How much sense it makes determines the improvisational modifier. Only if no-one can see how it makes any sense at all should you change your answer to 'no.' To rehash an old debate: do the Orlanthi have magical slingstones that explode on impact? Yes. Do they have monoliths at which they pray to summon storms? Yes. Do they call both of these things 'Thunderstones'? Yes. Easy, isn't it? == Extended contests are about decisions == I think extended contests are wonderful. The AP bidding mechanism can turn a boring set of dice rolls into an exciting, involved bit of roleplaying. It does this by allowing the player to make a decision each round, and for this decision to have a real game-mechanical effect. After seeing extended contests, does anyone really want to go back to the bad old days of having fights being endless repetitions of 'I attack,' 'I parry,' 'I attack,' 'I parry'? But it's the decision that's important, and this requires a bit of care in using extended contests. In particular: === Describe the action, then determine the AP bid === Each round in an extended contest, get the player to make a decision about how their character will act in that exchange. Once they've made that decision, they can then describe how the character will act. Once you've got that far, you can then fix the size of the AP bid. In my experience, doing things this way around keeps the contest interesting for all concerned. === If there aren't any decisions, don't use an extended contest === The corollary of the above is that extended contests shouldn't be used when there are no meaningful decisions to be made. I wouldn't use extended contests for things like arm wrestling or staring matches, no matter how climatic they are, simply because I can't envisage what different AP bids will look like. What's the real difference between 'I stare at her: 2AP bid' and 'I stare at her: 15AP bid'? Without decisions, extended contests degenerate into pure dice-rolling exercises; in those cases, use the simple contest mechanic and get it out of the way. As a half-way position, the symmetric nature of the results table means that extended contests work if only one party is bidding. If a PC is climbing a cliff, the cliff shouldn't make AP bids -- what decisions can it make? <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] 72d0ed6d904353c45025fc89f32d44db748b75e9 1663 1660 2008-02-08T14:19:26Z Neil 2 wikitext text/x-wiki Here are some pearls of wisdom I've learnt from narrating a HeroQuest campaign. In essence, my tips are: __TOC__ == Relax == It's very easy to get involved in debates over 'The One True Way' to play HeroQuest. Don't. Remember, HeroQuest is just a game, and a HeroQuest session is about you and a bunch of mates getting together to have fun. Don't worry too much about whether you're doing things according to the letter of the rulebook, or whether some obscure offering from Greg's wastebasket contrdicts what someone's just said. If people are having fun, it's a good game. == Learn to say 'Yes' == HeroQuest is a game of improvisation. There are no defined skill lists, feat lists, or whatever. PCs will improvise their abilities as they go along. You should allow, nay, encourage this. It can take some getting used to (it did for me), particularly if you're used to the more traditional games with fully defined abilities. Practice saying 'yes.' Next time a player asks you if a PC can use ability X to do Y, say 'yes' first, then worry about whether it makes sense. How much sense it makes determines the improvisational modifier. Only if no-one can see how it makes any sense at all should you change your answer to 'no.' To rehash an old debate: do the Orlanthi have magical slingstones that explode on impact? Yes. Do they have monoliths at which they pray to summon storms? Yes. Do they call both of these things 'Thunderstones'? Yes. Easy, isn't it? == Extended contests are about decisions == I think extended contests are wonderful. The AP bidding mechanism can turn a boring set of dice rolls into an exciting, involved bit of roleplaying. It does this by allowing the player to make a decision each round, and for this decision to have a real game-mechanical effect. After seeing extended contests, does anyone really want to go back to the bad old days of having fights being endless repetitions of 'I attack,' 'I parry,' 'I attack,' 'I parry'? But it's the decision that's important, and this requires a bit of care in using extended contests. In particular: === Describe the action, then determine the AP bid === Each round in an extended contest, get the player to make a decision about how their character will act in that exchange. Once they've made that decision, they can then describe how the character will act. Once you've got that far, you can then fix the size of the AP bid. In my experience, doing things this way around keeps the contest interesting for all concerned. === If there aren't any decisions, don't use an extended contest === The corollary of the above is that extended contests shouldn't be used when there are no meaningful decisions to be made. I wouldn't use extended contests for things like arm wrestling or staring matches, no matter how climatic they are, simply because I can't envisage what different AP bids will look like. What's the real difference between 'I stare at her: 2AP bid' and 'I stare at her: 15AP bid'? Without decisions, extended contests degenerate into pure dice-rolling exercises; in those cases, use the simple contest mechanic and get it out of the way. As a half-way position, the symmetric nature of the results table means that extended contests work if only one party is bidding. If a PC is climbing a cliff, the cliff shouldn't make AP bids -- what decisions can it make? <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] [[Category:HeroQuest]] 28c112ea075c375055ef74d5be4b5b96a2fbf23b 14 1507 1664 2008-02-08T14:21:40Z Neil 2 wikitext text/x-wiki Here are articles pertaining to the HeroQuest game system, as used for role-playing games set in [[:Category:Glorantha|Glorantha]]. <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> f5556e237724d29cff559e131776837ab415d52b 6 1509 1667 2008-02-15T22:42:43Z Neil 2 wikitext text/x-wiki da39a3ee5e6b4b0d3255bfef95601890afd80709 0 1 1668 1594 2008-02-16T08:11:31Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 35be4b583452e1b33b8832b82ba92710f29ab0e2 1681 1668 2008-05-16T18:50:58Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also written some notes on [[:Category:Server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Programs|released a few little programs]] as open source. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. bbfa328d23acd8738ea40a01b962aba0fd19b4ea 1682 1681 2008-05-16T18:55:53Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also written some notes on [[:Category:Server setup|setting up my various server machines]] (mainly for my own reference). == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 88dd44edcfdd337dbe6c9595dd2aab12be8735fd 0 1510 1669 2008-05-01T23:51:29Z Neil 2 wikitext text/x-wiki == Vanilla Everway == Everway is an extremely innovative role playing game. Its main features are the use of cards as randomisers, instead of dice, the use of Hermetic elements to describe characters, and its concentration on the inner voice of a character rather than its actions. I would like to apply Everway mechanics to Gloranthan role-playing, but there are problems with this (see below). In the meantime, here is my take on the cards that define the various realms in and around the Zola Fel valley: {| {{Prettytable}} |- |{{Hl3}}| Region |{{Hl3}}| Cards |- | '''Prax''' | ''Virtue:'' Summer (Energy)<br> ''Fault:'' The Peasant reversed (Lack of Vision)<br> ''Fate:'' Law (Order / Treachery) <br> ''Usurper:'' Wakboth (Chaos / Chaos Defeated)<br> |- | '''Sun County''' | ''Virtue:'' The Soldier (Duty) <br> ''Fault:'' The King reversed (Tyranny) <br> ''Fate:'' The Cockatrice (Corruption / Recovery) <br> ''Usurper:'' The Temple (Social Order / Social Disorder) |- | '''Pavisites''' | ''Virtue:'' Spring (New Growth) <br> ''Fault:'' Striking the Dragon's Tail (Underestimating the Challenge)<br> ''Fate:'' Fertility (Growth / Decline) <br> ''Usurper:'' The Troll (Violence / Harmony) |- | '''Pavic Orlanthi''' | ''Virtue:'' The Fool (Freedom) <br> ''Fault:'' The Eagle reversed (Thoughtlessness) <br> ''Fate:'' The Phoenix (Rebirth / Destruction)<br> ''Usurper:'' The Hearth (Kin / Kinstrife) |- | '''Lunars''' | ''Virtue:'' The Priestess (Understanding Mysteries)<br> ''Fault:'' The Satyr (Indulgence) <br> ''Fate:'' The Fish (The Soul Prevails / Hedonism) <br> ''Usurper:'' The Moon (Learning / Teaching) |} === Useful Links === Good starting points for Everway are: * [https://netfiles.uiuc.edu/rwb/www/gaming/everweb/ Rob Barrett's Everweb] * [http://www.gaslightpress.com/Everway/default.asp Gaslight Press], the current owners of Everway * Martin Teply has a [https://netfiles.uiuc.edu/rwb/www/gaming/everweb/Everway.List.pdf Reference Guide to the Fortune Deck], which I found very useful in constructing my own [[Praxian fortune deck]]. == Gloranthan Everway == While I think that the Everway mechanics have a great potential, and can lead to more colourful role-playing, there are major problem with applying Everway directly to Glorantha. For a start, the elements have different meanings in Glorantha from their meanings in Western Hermetic thought. {| {{Prettytable}} |- |{{Hl3}}| Everway Element |{{Hl3}}| Meaning |{{Hl3}}| Gloranthan Equivalent |- | Fire || Physical action || Storm, Sea? |- | Air || Intellect, perception || Fire/Sky |- | Water || Intuition, emotion || Darkness |- | Earth || Resilience || Earth |} And this is only true where the God Learners have been and made it true! For other cultures, such as the [[Category:Yellowtail|Praxians]], the Everway elements are completely up the spout. This means that it becomes difficult, if not impossible, to get players to understand the myths behind their actions if all the resonances between PC and world are completely different from the resonances between player and game. There's a further problem with applying Everway to cultures like the Praxians. There are implicit assumptions in Everway that the PCs will be exploring agrarian cultures, with egalitarian and meritocratic government, and that this is a Good Thing. But this doesn't fit other cultures. Sun County is based on the principle that egalitarianism is a Bad Thing, and everyone (rulers and ruled) agree. And why should Praxians be concerned with the harvest being brought in during the autumn, when they don't have either harvests or autumns! It is for these reasons that I have decided to develop my own version of Everway. In fact, I'll need versions, as each culture will have a different mindset which will need to be reflected in its own rule mechanics. At present, I'm only in the beginning stages of this process, but I'll be posting my notes here as they develop. Another approach was taken by John Hughes, who developed a [http://glorantha.temppeli.org/digest/gd2/1995.12/1696.html Tarot deck for the Far Point Orlanthi] (and see parts [http://glorantha.temppeli.org/digest/gd2/1995.12/1697.html 2] [http://glorantha.temppeli.org/digest/gd2/1995.12/1698.html 3] [http://glorantha.temppeli.org/digest/gd2/1995.12/1699.html 4] and [http://glorantha.temppeli.org/digest/gd2/1995.12/1700.html 5]). It's much more like a traditional Tarot deck than the Everway Fortune Deck. Loren Miller's [http://www.rpgmud.com/WorldBuilding/Mythopoets/tmm.html MythoPoet's Manual] was another great resource. === Praxian Everway === The first stage of this was to develop a metaphysical structure for the culture, in the form of the elements and meanings for that culture. Funnily enough, I started work on Praxians. You can read my thoughts on [[Praxian elements|how Praxians see the world and its elements]]. I have also written up the [[Praxian fortune deck]]. Of course, the Praxians wouldn't use cards -- they'd use entrails, thrown bones, the flight of birds across the sky, or somesuch to divine the future. But cards are useful for the players. <small>Everway is &copy; 1996-2007 by Gaslight Press. ''Everway'' is a trademark of Gaslight Press. None of the materials found herein are intended as challenges to the trademarks and/or copyrights of Gaslight Press. </small> [[Category:Glorantha]] [[Category:Everway]] 051e31c5fadd4b0f21160f14213d5d8679bfaeba 1670 1669 2008-05-01T23:52:35Z Neil 2 wikitext text/x-wiki == Vanilla Everway == Everway is an extremely innovative role playing game. Its main features are the use of cards as randomisers, instead of dice, the use of Hermetic elements to describe characters, and its concentration on the inner voice of a character rather than its actions. I would like to apply Everway mechanics to Gloranthan role-playing, but there are problems with this (see below). In the meantime, here is my take on the cards that define the various realms in and around the Zola Fel valley: {| {{Prettytable}} |- |{{Hl3}}| Region |{{Hl3}}| Cards |- | '''Prax''' | ''Virtue:'' Summer (Energy)<br> ''Fault:'' The Peasant reversed (Lack of Vision)<br> ''Fate:'' Law (Order / Treachery) <br> ''Usurper:'' Wakboth (Chaos / Chaos Defeated)<br> |- | '''Sun County''' | ''Virtue:'' The Soldier (Duty) <br> ''Fault:'' The King reversed (Tyranny) <br> ''Fate:'' The Cockatrice (Corruption / Recovery) <br> ''Usurper:'' The Temple (Social Order / Social Disorder) |- | '''Pavisites''' | ''Virtue:'' Spring (New Growth) <br> ''Fault:'' Striking the Dragon's Tail (Underestimating the Challenge)<br> ''Fate:'' Fertility (Growth / Decline) <br> ''Usurper:'' The Troll (Violence / Harmony) |- | '''Pavic Orlanthi''' | ''Virtue:'' The Fool (Freedom) <br> ''Fault:'' The Eagle reversed (Thoughtlessness) <br> ''Fate:'' The Phoenix (Rebirth / Destruction)<br> ''Usurper:'' The Hearth (Kin / Kinstrife) |- | '''Lunars''' | ''Virtue:'' The Priestess (Understanding Mysteries)<br> ''Fault:'' The Satyr (Indulgence) <br> ''Fate:'' The Fish (The Soul Prevails / Hedonism) <br> ''Usurper:'' The Moon (Learning / Teaching) |} === Useful Links === Good starting points for Everway are: * [https://netfiles.uiuc.edu/rwb/www/gaming/everweb/ Rob Barrett's Everweb] * [http://www.gaslightpress.com/Everway/default.asp Gaslight Press], the current owners of Everway * Martin Teply has a [https://netfiles.uiuc.edu/rwb/www/gaming/everweb/Everway.List.pdf Reference Guide to the Fortune Deck], which I found very useful in constructing my own [[Praxian fortune deck]]. == Gloranthan Everway == While I think that the Everway mechanics have a great potential, and can lead to more colourful role-playing, there are major problem with applying Everway directly to Glorantha. For a start, the elements have different meanings in Glorantha from their meanings in Western Hermetic thought. {| {{Prettytable}} |- |{{Hl3}}| Everway Element |{{Hl3}}| Meaning |{{Hl3}}| Gloranthan Equivalent |- | Fire || Physical action || Storm, Sea? |- | Air || Intellect, perception || Fire/Sky |- | Water || Intuition, emotion || Darkness |- | Earth || Resilience || Earth |} And this is only true where the God Learners have been and made it true! For other cultures, such as the [[:Category:Yellowtail|Praxians]], the Everway elements are completely up the spout. This means that it becomes difficult, if not impossible, to get players to understand the myths behind their actions if all the resonances between PC and world are completely different from the resonances between player and game. There's a further problem with applying Everway to cultures like the Praxians. There are implicit assumptions in Everway that the PCs will be exploring agrarian cultures, with egalitarian and meritocratic government, and that this is a Good Thing. But this doesn't fit other cultures. Sun County is based on the principle that egalitarianism is a Bad Thing, and everyone (rulers and ruled) agree. And why should Praxians be concerned with the harvest being brought in during the autumn, when they don't have either harvests or autumns! It is for these reasons that I have decided to develop my own version of Everway. In fact, I'll need versions, as each culture will have a different mindset which will need to be reflected in its own rule mechanics. At present, I'm only in the beginning stages of this process, but I'll be posting my notes here as they develop. Another approach was taken by John Hughes, who developed a [http://glorantha.temppeli.org/digest/gd2/1995.12/1696.html Tarot deck for the Far Point Orlanthi] (and see parts [http://glorantha.temppeli.org/digest/gd2/1995.12/1697.html 2] [http://glorantha.temppeli.org/digest/gd2/1995.12/1698.html 3] [http://glorantha.temppeli.org/digest/gd2/1995.12/1699.html 4] and [http://glorantha.temppeli.org/digest/gd2/1995.12/1700.html 5]). It's much more like a traditional Tarot deck than the Everway Fortune Deck. Loren Miller's [http://www.rpgmud.com/WorldBuilding/Mythopoets/tmm.html MythoPoet's Manual] was another great resource. === Praxian Everway === The first stage of this was to develop a metaphysical structure for the culture, in the form of the elements and meanings for that culture. Funnily enough, I started work on Praxians. You can read my thoughts on [[Praxian elements|how Praxians see the world and its elements]]. I have also written up the [[Praxian fortune deck]]. Of course, the Praxians wouldn't use cards -- they'd use entrails, thrown bones, the flight of birds across the sky, or somesuch to divine the future. But cards are useful for the players. <small>Everway is &copy; 1996-2007 by Gaslight Press. ''Everway'' is a trademark of Gaslight Press. None of the materials found herein are intended as challenges to the trademarks and/or copyrights of Gaslight Press. </small> [[Category:Glorantha]] [[Category:Everway]] 09fb2bb1298678d58f8b3f45c61f4697c4ca0b33 1671 1670 2008-05-02T00:06:56Z Neil 2 /* Gloranthan Everway */ wikitext text/x-wiki == Vanilla Everway == Everway is an extremely innovative role playing game. Its main features are the use of cards as randomisers, instead of dice, the use of Hermetic elements to describe characters, and its concentration on the inner voice of a character rather than its actions. I would like to apply Everway mechanics to Gloranthan role-playing, but there are problems with this (see below). In the meantime, here is my take on the cards that define the various realms in and around the Zola Fel valley: {| {{Prettytable}} |- |{{Hl3}}| Region |{{Hl3}}| Cards |- | '''Prax''' | ''Virtue:'' Summer (Energy)<br> ''Fault:'' The Peasant reversed (Lack of Vision)<br> ''Fate:'' Law (Order / Treachery) <br> ''Usurper:'' Wakboth (Chaos / Chaos Defeated)<br> |- | '''Sun County''' | ''Virtue:'' The Soldier (Duty) <br> ''Fault:'' The King reversed (Tyranny) <br> ''Fate:'' The Cockatrice (Corruption / Recovery) <br> ''Usurper:'' The Temple (Social Order / Social Disorder) |- | '''Pavisites''' | ''Virtue:'' Spring (New Growth) <br> ''Fault:'' Striking the Dragon's Tail (Underestimating the Challenge)<br> ''Fate:'' Fertility (Growth / Decline) <br> ''Usurper:'' The Troll (Violence / Harmony) |- | '''Pavic Orlanthi''' | ''Virtue:'' The Fool (Freedom) <br> ''Fault:'' The Eagle reversed (Thoughtlessness) <br> ''Fate:'' The Phoenix (Rebirth / Destruction)<br> ''Usurper:'' The Hearth (Kin / Kinstrife) |- | '''Lunars''' | ''Virtue:'' The Priestess (Understanding Mysteries)<br> ''Fault:'' The Satyr (Indulgence) <br> ''Fate:'' The Fish (The Soul Prevails / Hedonism) <br> ''Usurper:'' The Moon (Learning / Teaching) |} === Useful Links === Good starting points for Everway are: * [https://netfiles.uiuc.edu/rwb/www/gaming/everweb/ Rob Barrett's Everweb] * [http://www.gaslightpress.com/Everway/default.asp Gaslight Press], the current owners of Everway * Martin Teply has a [https://netfiles.uiuc.edu/rwb/www/gaming/everweb/Everway.List.pdf Reference Guide to the Fortune Deck], which I found very useful in constructing my own [[Praxian fortune deck]]. == Gloranthan Everway == While I think that the Everway mechanics have a great potential, and can lead to more colourful role-playing, there are major problem with applying Everway directly to Glorantha. For a start, the elements have different meanings in Glorantha from their meanings in Western Hermetic thought. {| {{Prettytable}} |- |{{Hl3}}| Everway Element |{{Hl3}}| Meaning |{{Hl3}}| Gloranthan Equivalent |- | Fire || Physical action || Storm, Sea? |- | Air || Intellect, perception || Fire/Sky |- | Water || Intuition, emotion || Darkness |- | Earth || Resilience || Earth |} And this is only true where the God Learners have been and made it true! For other cultures, such as the [[:Category:Yellowtail|Praxians]], the Everway elements are completely up the spout. This means that it becomes difficult, if not impossible, to get players to understand the myths behind their actions if all the resonances between PC and world are completely different from the resonances between player and game. There's a further problem with applying Everway to cultures like the Praxians. There are implicit assumptions in Everway that the PCs will be exploring agrarian cultures, with egalitarian and meritocratic government, and that this is a Good Thing. But this doesn't fit other cultures. Sun County is based on the principle that egalitarianism is a Bad Thing, and everyone (rulers and ruled) agree. And why should Praxians be concerned with the harvest being brought in during the autumn, when they don't have either harvests or autumns! It is for these reasons that I have decided to develop my own version of Everway. In fact, I'll need versions, as each culture will have a different mindset which will need to be reflected in its own rule mechanics. At present, I'm only in the beginning stages of this process, but I'll be posting my notes here as they develop. Another approach was taken by John Hughes, who developed a [http://glorantha.temppeli.org/digest/gd2/1995.12/1696.html Tarot deck for the Far Point Orlanthi] (and see parts [http://glorantha.temppeli.org/digest/gd2/1995.12/1697.html 2], [http://glorantha.temppeli.org/digest/gd2/1995.12/1698.html 3], [http://glorantha.temppeli.org/digest/gd2/1995.12/1699.html 4], and [http://glorantha.temppeli.org/digest/gd2/1995.12/1700.html 5]). It's much more like a traditional Tarot deck than the Everway Fortune Deck. Loren Miller's [http://www.rpgmud.com/WorldBuilding/Mythopoets/tmm.html MythoPoet's Manual] was another great resource. === Praxian Everway === The first stage of this was to develop a metaphysical structure for the culture, in the form of the elements and meanings for that culture. Funnily enough, I started work on Praxians. You can read my thoughts on [[Praxian elements|how Praxians see the world and its elements]]. I have also written up the [[Praxian fortune deck]]. Of course, the Praxians wouldn't use cards -- they'd use entrails, thrown bones, the flight of birds across the sky, or somesuch to divine the future. But cards are useful for the players. <small>Everway is &copy; 1996-2007 by Gaslight Press. ''Everway'' is a trademark of Gaslight Press. None of the materials found herein are intended as challenges to the trademarks and/or copyrights of Gaslight Press. </small> [[Category:Glorantha]] [[Category:Everway]] 08384b1fbcf5466acc86e7f3dc83d51baad83bc3 1673 1671 2008-05-02T00:15:15Z Neil 2 /* Vanilla Everway */ wikitext text/x-wiki == Vanilla Everway == Everway is an extremely innovative role playing game. Its main features are the use of cards as randomisers, instead of dice, the use of Hermetic elements to describe characters, and its concentration on the inner voice of a character rather than its actions. I would like to apply Everway mechanics to Gloranthan role-playing, but there are problems with this ([[#Gloranthan Everway|see below]]). In the meantime, here is my take on the cards that define the various realms in and around the Zola Fel valley: {| {{Prettytable}} |- |{{Hl3}}| Region |{{Hl3}}| Cards |- | '''Prax''' | ''Virtue:'' Summer (Energy)<br> ''Fault:'' The Peasant reversed (Lack of Vision)<br> ''Fate:'' Law (Order / Treachery) <br> ''Usurper:'' Wakboth (Chaos / Chaos Defeated)<br> |- | '''Sun County''' | ''Virtue:'' The Soldier (Duty) <br> ''Fault:'' The King reversed (Tyranny) <br> ''Fate:'' The Cockatrice (Corruption / Recovery) <br> ''Usurper:'' The Temple (Social Order / Social Disorder) |- | '''Pavisites''' | ''Virtue:'' Spring (New Growth) <br> ''Fault:'' Striking the Dragon's Tail (Underestimating the Challenge)<br> ''Fate:'' Fertility (Growth / Decline) <br> ''Usurper:'' The Troll (Violence / Harmony) |- | '''Pavic Orlanthi''' | ''Virtue:'' The Fool (Freedom) <br> ''Fault:'' The Eagle reversed (Thoughtlessness) <br> ''Fate:'' The Phoenix (Rebirth / Destruction)<br> ''Usurper:'' The Hearth (Kin / Kinstrife) |- | '''Lunars''' | ''Virtue:'' The Priestess (Understanding Mysteries)<br> ''Fault:'' The Satyr (Indulgence) <br> ''Fate:'' The Fish (The Soul Prevails / Hedonism) <br> ''Usurper:'' The Moon (Learning / Teaching) |} === Useful Links === Good starting points for Everway are: * [https://netfiles.uiuc.edu/rwb/www/gaming/everweb/ Rob Barrett's Everweb] * [http://www.gaslightpress.com/Everway/default.asp Gaslight Press], the current owners of Everway * Martin Teply has a [https://netfiles.uiuc.edu/rwb/www/gaming/everweb/Everway.List.pdf Reference Guide to the Fortune Deck], which I found very useful in constructing my own [[Praxian fortune deck]]. == Gloranthan Everway == While I think that the Everway mechanics have a great potential, and can lead to more colourful role-playing, there are major problem with applying Everway directly to Glorantha. For a start, the elements have different meanings in Glorantha from their meanings in Western Hermetic thought. {| {{Prettytable}} |- |{{Hl3}}| Everway Element |{{Hl3}}| Meaning |{{Hl3}}| Gloranthan Equivalent |- | Fire || Physical action || Storm, Sea? |- | Air || Intellect, perception || Fire/Sky |- | Water || Intuition, emotion || Darkness |- | Earth || Resilience || Earth |} And this is only true where the God Learners have been and made it true! For other cultures, such as the [[:Category:Yellowtail|Praxians]], the Everway elements are completely up the spout. This means that it becomes difficult, if not impossible, to get players to understand the myths behind their actions if all the resonances between PC and world are completely different from the resonances between player and game. There's a further problem with applying Everway to cultures like the Praxians. There are implicit assumptions in Everway that the PCs will be exploring agrarian cultures, with egalitarian and meritocratic government, and that this is a Good Thing. But this doesn't fit other cultures. Sun County is based on the principle that egalitarianism is a Bad Thing, and everyone (rulers and ruled) agree. And why should Praxians be concerned with the harvest being brought in during the autumn, when they don't have either harvests or autumns! It is for these reasons that I have decided to develop my own version of Everway. In fact, I'll need versions, as each culture will have a different mindset which will need to be reflected in its own rule mechanics. At present, I'm only in the beginning stages of this process, but I'll be posting my notes here as they develop. Another approach was taken by John Hughes, who developed a [http://glorantha.temppeli.org/digest/gd2/1995.12/1696.html Tarot deck for the Far Point Orlanthi] (and see parts [http://glorantha.temppeli.org/digest/gd2/1995.12/1697.html 2], [http://glorantha.temppeli.org/digest/gd2/1995.12/1698.html 3], [http://glorantha.temppeli.org/digest/gd2/1995.12/1699.html 4], and [http://glorantha.temppeli.org/digest/gd2/1995.12/1700.html 5]). It's much more like a traditional Tarot deck than the Everway Fortune Deck. Loren Miller's [http://www.rpgmud.com/WorldBuilding/Mythopoets/tmm.html MythoPoet's Manual] was another great resource. === Praxian Everway === The first stage of this was to develop a metaphysical structure for the culture, in the form of the elements and meanings for that culture. Funnily enough, I started work on Praxians. You can read my thoughts on [[Praxian elements|how Praxians see the world and its elements]]. I have also written up the [[Praxian fortune deck]]. Of course, the Praxians wouldn't use cards -- they'd use entrails, thrown bones, the flight of birds across the sky, or somesuch to divine the future. But cards are useful for the players. <small>Everway is &copy; 1996-2007 by Gaslight Press. ''Everway'' is a trademark of Gaslight Press. None of the materials found herein are intended as challenges to the trademarks and/or copyrights of Gaslight Press. </small> [[Category:Glorantha]] [[Category:Everway]] d1984c689aa695a1487484a92f850c50d01dc6f4 0 1511 1672 2008-05-02T00:13:29Z Neil 2 wikitext text/x-wiki In my continuing attempts to create a version of the Everway rules for use in a Praxian context, I re-examining the Praxian metaphysics and cultural background. I have two main objectives in doing this: firstly, I want to create a playable version of Everway tailored for Praxians; secondly, I want to explore the Praxian culture from the Praxian point of view, hopefully making them psychologically more distinct. This process is completed with the [[Praxian fortune deck]]. In Everway, characters are described in terms of the four hermetic elements: Fire, Air, Earth, and Water. Different scores in these areas represent different abilities. However, these elements have different meanings and associations from the traditional Gloranthan/God Learner ones. In addition, the Praxians don't have the same God Learner influenced heritage of the Orlanthi, so there is no reason for the Praxians to pay any real attention to the 'standard' Gloranthan five elements. == The Elements == Instead, I decided to start from the basics of their world view and asked what the Praxians thought important. These things are represented by their gods. This gave the following basic descriptions: {| {{Prettytable}} |- |{{Hl3}}| God |{{Hl3}}| Element |{{Hl3}}| General meaning |- | Waha || Authority || Authority, intellect, coldness |- | Eiritha || Beast || Resilience, nurturing, stubborness |- | Storm Bull || Storm || Activity, energy, uncontrollability |- | Daka Fal || Spirit || Magic, intuition, whimsicality |} * Eiritha is the source of all bounty and wealth. She survived the God's War by staying. * Storm Bull came and changed things, just because he could. * Daka Fal understands the mysteries of the other world. * Waha shows us how to control this world, and shows us what is right. The elements are normally portrayed in a circle (representing the cycle of Rebirth and Death), divided into four parts: {| |- | || Authority || |- | Spirit || || Storm |- | || Beast || |} Beast is at the bottom, as all things rest on the herds. Spirit is on the left, betraying a draconic influence from the EWF. Storm is on the right, as it is the right hand that performs actions. Authority is on top, as a man rides atop a beast. Each of these elements governs a particular type of ability and is associated with a particular personality type: {| {{Prettytable}} |- |{{Hl3}}| Element |{{Hl3}}| Actions |{{Hl3}}| Personality |- | Authority | Perception, deduction, willpower, martial arts | Analytical, calm, taciturn, intellectual, distant, perceptive, dispassionate |- | Beast | Resistance, endurance, physical robustness | Honest, dependable, trustworthy, stubborn, frank, reliable, comforting, tough |- | Storm | Physical activity, gumption, passions | Active, violent, energetic, impatient, erratic, impetuous, (over-)confident |- | Spirit | Intuition, magical perception, empathy | Sensitive, intuitive, emotional, diplomatic, whimsical, sentimental, maudlin, idealistic |} == Implications of these Elemental choices == While these qualities for the elements are close to the standard Everway hermetic version, there are a couple of things to note. The first of these is to do with combat, which betrays a Kralori influence on the Praxian mindset. Waha imposes his will, his intellect, on the world. Killing, whether of beasts or men, should be an unemotional activity, merely part of the cycle of Life and Rebirth. Mental discipline is required to demonstrate the skills of the butcher. This attitude has lead to the development of formalised weapon techniques, almost becoming martial arts. This is most obviously recognised in the Morokanth, who have greatly developed their unarmed combat techniques in this way. This also acts as another reason why Storm Bullies are despied: they give in to their passions during combat, and cannot achieve the inner calm that is required by the Wahahim. Charisma, communication, and diplomacy rely on the Spirit. High Authority makes Khans capable but taciturn leaders. Khans would make good generals if they organised themselves in anything bigger than a clan. The Waha Spirits of Law become Spirits of Authority. They lose their specialisation against Chaos, and instead can be used against those that have broken the Covenant of Waha (see [[why Chaos is a cop-out]] for why I made this change). === Combinations of Elements === Combinations of the elements also have meanings for a character, showing how different parts of the psyche act together. * Storm and Beast together represent Power, the ability to make changes in the world. Storm makes the changes while Beast carries them through. * Authority and Spirit represent Wisdom, the ability to judge which changes should be made. Authority shows the way of Waha, while Spirit shows intuition. * Beast and Spirit show the Passive side, how changes are dealt with. Beast resists change, while Spirit adapts. * Authority and Storm are the Active elements. Authority is the imposition of one's will on the world, while Storm does new things. * Authority and Beast, the vertical axis, represent conservatism, stability, and peace. This is generally considered Good. * Storm and Spirit, the horizontal axis, represent innovation, eccentricity, and change. This is generally considered Bad. === Miscellaneous implications === The association of elements can also be taken further, and each of the Great Tribes can be associated with an element: {| {{Prettytable}} |- |{{Hl3}}| Element |{{Hl3}}| Tribe |- | Authority || High Llama |- | Beast || Bison |- | Storm || Impala |- | Spirit || Morokanth |} The tribe associated with each element emphasises that element more than the others. For instance, the Impala tribe (particularly the bachelors) embody the Storm qualities of violent, uncontrolled activity. The Sable tribe does not emphasise an element, as clans try to maintain a balance between each of the elements. This has some implications as to how the differences between the tribes come about. Morokanth are labelled by the God Learners as darkness related, as they focus on the spiritual aspects of the world. Through this, they fell foul of the God Learners' equivalence between spirits and darkness. However, they are astute enough to exploit this supposed correspondence, and tend to have an increased facility with Darkness spirits and magic in God Learner influenced areas. The High Llamas take pride in their intellect and perceptivness, which is why they are also haughty. In combat, they rely on superior tactics to make up for their lack on numbers. The Bison tribe, not Sable, is matriarchal. Sables rule with committee, with Eiritha queens generally dominating the discussions, as herd welfare is normally paramount. It also explains why Bison riders are so tenacious in combat. The Impala tribe will have a greater proportion of Storm Bullies than listed, particularly among the bachelor males. The Sable tribe emphasises balance and harmony between elements, and hence were more susceptible to Lunar advances than other Praxians (of course, the chance of being on the winning side at Moonbroth also helped). The correspondence can also be extended to the Three Feathered Rivals (though less convincingly): {| {{Prettytable}} |- |{{Hl3}}| Element |{{Hl3}}| Rival |- | Authority || Sun Hawk |- | Beast || Condor (though not so much since Eiritha was buried: being buried makes life difficult for a bird) |- | Storm || Thunderbird |- | Spirit || Raven |} and perhaps also parts of the body: {| {{Prettytable}} |- |{{Hl3}}| Element |{{Hl3}}| Rival |- | Authority || Head, eyes |- | Beast || Body (liver?), genitals |- | Storm || Right arm and side |- | Spirit || Left arm and side |} == Want to know more? == More information about Prax and the Praxians can be found in the [http://www.glorantha.com/library/ Lhankor Mhy Research Library], particularly: * The Praxian 'What my Father Told Me' (in [http://www.heroquest-rpg.com/products/3001.html HeroQuest Voices]) * Words from the Tribal Shaman (in [http://www.heroquest-rpg.com/products/3001.html HeroQuest Voices]) * [http://www.glorantha.com/library/religions/cop-nomadgod.html Praxian Gods * [http://www.glorantha.com/library/religions/cult-waha.html The cult of Waha] * [http://www.glorantha.com/library/religions/cult-eiritha.html The cult of Eiritha] * [http://www.glorantha.com/library/religions/cult-dakafal.html The cult of Daka Fal] * [http://www.glorantha.com/library/religions/cult-stormbull.html The cult of Storm Bull] <small>Everway is &copy; 1996-2007 by Gaslight Press. ''Everway'' is a trademark of Gaslight Press. None of the materials found herein are intended as challenges to the trademarks and/or copyrights of Gaslight Press. </small> [[Category:Glorantha]] [[Category:Everway]] d2c24f87088ef6ba3bcd484db6672d2e60a53fdc 1674 1672 2008-05-02T00:25:53Z Neil 2 wikitext text/x-wiki In my continuing attempts to create a version of the Everway rules for use in a Praxian context, I re-examining the Praxian metaphysics and cultural background. I have two main objectives in doing this: firstly, I want to create a playable version of Everway tailored for Praxians; secondly, I want to explore the Praxian culture from the Praxian point of view, hopefully making them psychologically more distinct. This process is completed with the [[Praxian fortune deck]]. In Everway, characters are described in terms of the four hermetic elements: Fire, Air, Earth, and Water. Different scores in these areas represent different abilities. However, these elements have different meanings and associations from the traditional Gloranthan/God Learner ones. In addition, the Praxians don't have the same God Learner influenced heritage of the Orlanthi, so there is no reason for the Praxians to pay any real attention to the 'standard' Gloranthan five elements. == The Elements == Instead, I decided to start from the basics of their world view and asked what the Praxians thought important. These things are represented by their gods. This gave the following basic descriptions: {| {{Prettytable}} |- |{{Hl3}}| God |{{Hl3}}| Element |{{Hl3}}| General meaning |- | Waha || Authority || Authority, intellect, coldness |- | Eiritha || Beast || Resilience, nurturing, stubborness |- | Storm Bull || Storm || Activity, energy, uncontrollability |- | Daka Fal || Spirit || Magic, intuition, whimsicality |} * Eiritha is the source of all bounty and wealth. She survived the God's War by staying. * Storm Bull came and changed things, just because he could. * Daka Fal understands the mysteries of the other world. * Waha shows us how to control this world, and shows us what is right. The elements are normally portrayed in a circle (representing the cycle of Rebirth and Death), divided into four parts: {| |- | || Authority || |- | Spirit || || Storm |- | || Beast || |} Beast is at the bottom, as all things rest on the herds. Spirit is on the left, betraying a draconic influence from the EWF. Storm is on the right, as it is the right hand that performs actions. Authority is on top, as a man rides atop a beast. Each of these elements governs a particular type of ability and is associated with a particular personality type: {| {{Prettytable}} |- |{{Hl3}}| Element |{{Hl3}}| Actions |{{Hl3}}| Personality |- | Authority | Perception, deduction, willpower, martial arts | Analytical, calm, taciturn, intellectual, distant, perceptive, dispassionate |- | Beast | Resistance, endurance, physical robustness | Honest, dependable, trustworthy, stubborn, frank, reliable, comforting, tough |- | Storm | Physical activity, gumption, passions | Active, violent, energetic, impatient, erratic, impetuous, (over-)confident |- | Spirit | Intuition, magical perception, empathy | Sensitive, intuitive, emotional, diplomatic, whimsical, sentimental, maudlin, idealistic |} == Implications of these Elemental choices == While these qualities for the elements are close to the standard Everway hermetic version, there are a couple of things to note. The first of these is to do with combat, which betrays a Kralori influence on the Praxian mindset. Waha imposes his will, his intellect, on the world. Killing, whether of beasts or men, should be an unemotional activity, merely part of the cycle of Life and Rebirth. Mental discipline is required to demonstrate the skills of the butcher. This attitude has lead to the development of formalised weapon techniques, almost becoming martial arts. This is most obviously recognised in the Morokanth, who have greatly developed their unarmed combat techniques in this way. This also acts as another reason why Storm Bullies are despied: they give in to their passions during combat, and cannot achieve the inner calm that is required by the Wahahim. Charisma, communication, and diplomacy rely on the Spirit. High Authority makes Khans capable but taciturn leaders. Khans would make good generals if they organised themselves in anything bigger than a clan. The Waha Spirits of Law become Spirits of Authority. They lose their specialisation against Chaos, and instead can be used against those that have broken the Covenant of Waha (see [[why Chaos is a cop-out]] for why I made this change). === Combinations of Elements === Combinations of the elements also have meanings for a character, showing how different parts of the psyche act together. * Storm and Beast together represent Power, the ability to make changes in the world. Storm makes the changes while Beast carries them through. * Authority and Spirit represent Wisdom, the ability to judge which changes should be made. Authority shows the way of Waha, while Spirit shows intuition. * Beast and Spirit show the Passive side, how changes are dealt with. Beast resists change, while Spirit adapts. * Authority and Storm are the Active elements. Authority is the imposition of one's will on the world, while Storm does new things. * Authority and Beast, the vertical axis, represent conservatism, stability, and peace. This is generally considered Good. * Storm and Spirit, the horizontal axis, represent innovation, eccentricity, and change. This is generally considered Bad. === Miscellaneous implications === The association of elements can also be taken further, and each of the Great Tribes can be associated with an element: {| {{Prettytable}} |- |{{Hl3}}| Element |{{Hl3}}| Tribe |- | Authority || High Llama |- | Beast || Bison |- | Storm || Impala |- | Spirit || Morokanth |} The tribe associated with each element emphasises that element more than the others. For instance, the Impala tribe (particularly the bachelors) embody the Storm qualities of violent, uncontrolled activity. The Sable tribe does not emphasise an element, as clans try to maintain a balance between each of the elements. This has some implications as to how the differences between the tribes come about. Morokanth are labelled by the God Learners as darkness related, as they focus on the spiritual aspects of the world. Through this, they fell foul of the God Learners' equivalence between spirits and darkness. However, they are astute enough to exploit this supposed correspondence, and tend to have an increased facility with Darkness spirits and magic in God Learner influenced areas. The High Llamas take pride in their intellect and perceptivness, which is why they are also haughty. In combat, they rely on superior tactics to make up for their lack on numbers. The Bison tribe, not Sable, is matriarchal. Sables rule with committee, with Eiritha queens generally dominating the discussions, as herd welfare is normally paramount. It also explains why Bison riders are so tenacious in combat. The Impala tribe will have a greater proportion of Storm Bullies than listed, particularly among the bachelor males. The Sable tribe emphasises balance and harmony between elements, and hence were more susceptible to Lunar advances than other Praxians (of course, the chance of being on the winning side at Moonbroth also helped). The correspondence can also be extended to the Three Feathered Rivals (though less convincingly): {| {{Prettytable}} |- |{{Hl3}}| Element |{{Hl3}}| Rival |- | Authority || Sun Hawk |- | Beast || Condor (though not so much since Eiritha was buried: being buried makes life difficult for a bird) |- | Storm || Thunderbird |- | Spirit || Raven |} and perhaps also parts of the body: {| {{Prettytable}} |- |{{Hl3}}| Element |{{Hl3}}| Rival |- | Authority || Head, eyes |- | Beast || Body (liver?), genitals |- | Storm || Right arm and side |- | Spirit || Left arm and side |} == Want to know more? == More information about Prax and the Praxians can be found in the [http://www.glorantha.com/library/ Lhankor Mhy Research Library], particularly: * The Praxian 'What my Father Told Me' (in [http://www.heroquest-rpg.com/products/3001.html HeroQuest Voices]) * Words from the Tribal Shaman (in [http://www.heroquest-rpg.com/products/3001.html HeroQuest Voices]) * [http://www.glorantha.com/library/religions/cop-nomadgod.html Praxian Gods * [http://www.glorantha.com/library/religions/cult-waha.html The cult of Waha] * [http://www.glorantha.com/library/religions/cult-eiritha.html The cult of Eiritha] * [http://www.glorantha.com/library/religions/cult-dakafal.html The cult of Daka Fal] * [http://www.glorantha.com/library/religions/cult-stormbull.html The cult of Storm Bull] <small>Everway is &copy; 1996-2007 by Gaslight Press. ''Everway'' is a trademark of Gaslight Press. None of the materials found herein are intended as challenges to the trademarks and/or copyrights of Gaslight Press. </small> [[Category:Glorantha]] [[Category:Yellowtail]] [[Category:Everway]] fed1520f09d9467779f0fe0699ebb58a7d355f99 0 1512 1675 2008-05-02T00:27:02Z Neil 2 wikitext text/x-wiki These are the cards I developed for the fortune deck for my [[Gloranthan Everway#Praxian Everway|Praxian Everway]] rules variant. To make sense of this, you'll probably need to read the [[Praxian elements]] page too. Some 'designer notes' are at the bottom of this page. {| {{Prettytable}} |- |{{Hl3}}| Card |{{Hl3}}| Meaning |{{Hl3}}| Reversed meaning |{{Hl3}}| Associations |- !colspan="4"| The Leaders |- | Khan || Authority || Solitude || Authority (leadership) |- | Queen || Solidarity || Undiscerning || Beast (nurturing) |- | Storm Khan || Violence || Unreasoning || Storm (aggression) |- | Shaman || Intuition || Lack of connection to world || Spirit (understanding mysteries) |- !colspan="4"| The Nations |- | Sable || Balance || Self-centred || All elements in balance |- | Morokanth || Innovation || Impracticality || Spirit (intuition) |- | High Llama || Perception || Haughty || Authority (perception) |- | Bison || Resilience || Unyielding || Beast (robustness) |- | Impala || Activity || Restlessness || Storm (activity) |- !colspan="4"| The Places |- | Plateau of Statues || Meaninglessness || Seeing hidden truths || Failure of spirit |- | Plains || Stagnation || Freedom || Failure of storm |- | Serpent || Discord, rapid change || Flexibility || Failure of authority |- | Oasis || Weakness || Recuperation || Failure of beast |- !colspan="4"| The Feathered Rivals |- | Raven || Impetuosity || Achieving the unexpected || Excess of spirit |- | Thunderbird || Transgressing boundaries || Ignoring constraints || Excess of storm |- | Sun Hawk || Hubris || Taking necessary action || Excess of authority |- | Condor || Dogmatism || Self-reliance || Excess of beast |- !colspan="4"| The Outlanders |- | Genert || Hopelessness || Paradise found || All elements in balance |- | Dragon || Devastation || Overpowering || Active (auth + storm) |- | Teshnans || Compliance, retreat || Eventual return || Passive (beast + spirit) |- | Westeners (Orlanthi) || Incessant activity || Serendipity || Power (storm + beast) |- | Kralori || Meditative inactivity || Careful investigation || Wisdom (auth + spirit) |- | Hsunchen || Labour in vain || Success of tradition || Conservative (auth + beast) |- | Pentans || Breaking the Covenant || Reflecting the others power || Change (storm + spirit) |- | Oasis farmers || Defeat || Hope in bleakness || None |- !colspan="4"| The Spirits |- | Bright Treasure || Realising limits || Over-preparation || Authority over Spirit |- | Bronze Treasure || Concealment || Treachery || Spirit over Storm |- | Malia || Corruption || Test that strengthens || Spirit over Beast |- | Dark Eater || Slavery || Over-reliance on others || Beast over Storm |- | Wildfire (Oakfed) || Exuberance released || Controlling wildness || Storm over Authority |- | Foundchild || Successful simplicity || Lack of imagination || Storm over Spirit |- !colspan="4"| The Universals |- | The Warband || Going outside || Exposure || Authority over Beast |- | The Herd || Home, safety || Indulgence || Beast over Authority |- !colspan="4"| Life Stages |- | The child || New life || Fragility of life || Beast over Spirit |- | The youth || Discoveries || Mistakes || Storm over Beast |- | The adult || Purposeful activity || Constraints || Authority over Storm |- | The elder || Holistic awareness || Inability to act || Spirit over Authority |} == Designer Notes == Developing the fortune deck was rather more difficult than it first appeared. I tried several combinations and sets of cards and ideas, and nothing seemed to really work. Then I had the realisation that, although the fortune deck appears to be a divination tool, it is actually a game adjudication device. This allowed me to rebuild the deck around the notion of the factors that a GM could use to influence their decision on the success of an action. And that is what is above. Despite this focus on the game rather than the world, I do think that the fortune deck is a pretty fair representation of the important things in a Praxian's life and cognitive map. The topics embedded in this deck are the things that he (or she) knows and is concerned with. To that end, I think it gives an important insight into the Praxian mind. <small>Everway is &copy; 1996-2007 by Gaslight Press. ''Everway'' is a trademark of Gaslight Press. None of the materials found herein are intended as challenges to the trademarks and/or copyrights of Gaslight Press. </small> [[Category:Glorantha]] [[Category:Yellowtail]] [[Category:Everway]] dafabccae0590b7e2b00fe9fb396086f91cddea3 14 1513 1676 2008-05-02T00:28:00Z Neil 2 wikitext text/x-wiki Pages for my conversion of Everway to Glorantha. [[Category:Glorantha]] [[Category:RPGs]] 2834cad1e8ebcf138f2c7913088aa244ebd57fb3 1677 1676 2008-05-02T00:37:32Z Neil 2 wikitext text/x-wiki Pages for my conversion of Everway to Glorantha. <small>Everway is &copy; 1996-2007 by Gaslight Press. Everway is a trademark of Gaslight Press. None of the materials found herein are intended as challenges to the trademarks and/or copyrights of Gaslight Press.</small> [[Category:Glorantha]] [[Category:RPGs]] 785c2521a907281afe517dcba144e630a6e35b15 14 1456 1678 1617 2008-05-08T07:39:45Z Neil 2 /* External sites */ Added RedBrick links wikitext text/x-wiki Blue Planet is a role-playing game of mostly hard science fiction. The publishers are [http://www.biohazardgames.com Biohazard Games]. The game emphasises issues around ecological awareness and biotechnology. This page contains a few original articles, see below. === External sites === There are few Blue Planet websites out there, but their quality more than makes up for a lack of quantity. [http://www.biohazardgames.com Biohazard Games] are the original creators of Blue Planet (though Fantasy Flight Games did the publication for BP v2), and their site contains all sorts of useful and interesting snippets. I find it easier to find things by starting from [http://www.biohazardgames.com/sitemap.html their site map]. [http://www.ardanyan.de/redbrick/ RedBrick] now publish the Blue Planet v2 books and they've set up a [http://www.blueplanet-rpg.com Blue Planet website]. The [http://seahawk.110mb.com/ Dark Water] site has links to a great number of Blue Planet websites. The site's [http://seahawk.110mb.com/misc/index.php 'miscellaneous' page] has archived copies of two great resources, the OceanView periodical and Mike Z's Blue Planet notebook. OceanView really brings the world to life, and gives that sense that there's a lot more going on than just what your little group is doing. Mike Z's notebook contains all sorts of useful snippets of information and quick-reference tables. SeaHawk also [http://www.geocities.com/seahawk2199/BluePlanet/bpsupport.html has some larger (PDF) files on Geocities], including the first chapter of the Player's Guide, an excellent introduction to the game and the world. [[Category:RPGs]] af1eb8540a0c8378fd2b85a816fa3d4ec9b38433 1680 1678 2008-05-08T07:46:10Z Neil 2 /* External sites */ wikitext text/x-wiki Blue Planet is a role-playing game of mostly hard science fiction. The publishers are [http://www.biohazardgames.com Biohazard Games]. The game emphasises issues around ecological awareness and biotechnology. This page contains a few original articles, see below. === External sites === There are few Blue Planet websites out there, but their quality more than makes up for a lack of quantity. [http://www.biohazardgames.com Biohazard Games] are the original creators of Blue Planet (though Fantasy Flight Games did the publication for BP v2), and their site contains all sorts of useful and interesting snippets. I find it easier to find things by starting from [http://www.biohazardgames.com/sitemap.html their site map]. [http://www.ardanyan.de/redbrick/ RedBrick] now publish the Blue Planet v2 books and they've set up a [http://www.blueplanet-rpg.com Blue Planet website]. The [http://seahawk.110mb.com/ Dark Water] site has links to a great number of Blue Planet websites. The site's [http://seahawk.110mb.com/misc/index.php 'miscellaneous' page] has archived copies of two great resources, the OceanView periodical and Mike Z's Blue Planet notebook. OceanView really brings the world to life, and gives that sense that there's a lot more going on than just what your little group is doing. Mike Z's notebook contains all sorts of useful snippets of information and quick-reference tables. SeaHawk also [http://www.geocities.com/seahawk2199/BluePlanet/bpsupport.html has some larger (PDF) files on Geocities], including the first chapter of the Player's Guide, an excellent introduction to the game and the world. Finally, the [http://bpwiki.beachheadonline.com/index.php5?title=Main_Page Blue Planet wiki] has a growing number of articles, both official and fan-contributed. [[Category:RPGs]] ddd6a30691a706cdea60476dbdf03982009c01fd 0 1481 1679 1625 2008-05-08T07:43:50Z Neil 2 wikitext text/x-wiki I use a few house rules for the Synergy system, as used in [[:Category:Blue Planet|Blue Planet]]. They are all minor, and have very little effect on the already lightweight Synergy game system. See also the [http://bpwiki.beachheadonline.com/index.php5?title=Errata#Version_2 errata] on the [http://bpwiki.beachheadonline.com/index.php5?title=Main_Page Blue Planet wiki]. == Character Generation == I made a few restrictions on the character generation choices available to the players. * Characters are at the Exceptional power level, apart from having Everyday attributtes (i.e. sum to zero) * There are no systemic osmoform aquaforms (squids): gills on a mammal would cause virtually instant and terminal hypothermia. Diving reflex aquaforms, however, are perfectly believable. * Hybrids look far more like normal humans than how they are presented in the rules: genetic modification to increase muscle mass, say, will have no effect on the development of, say, the nose. Silvas tend to look like people with acromegaly, with large jaws and hands and increased muscle mass. Cats are thin, nervous, and constantly jumpy. == Open-ended dice rolls == When making a task attempt, natural rolls of '1' and '10' are open ended. If the lowest die rolled is a '1', add 10 to the Target Number and roll again (but only reroll the dice that show a '1'). If the lowest die rolled is a '10', subtract 10 from the Target number and roll again. Multiple consecutive rolls of '1' or '10' keep moving the Target Number. This mechanism allows attempts at tasks that, through various factors, have negative target numbers. :(Inspired by Mike Z's house rule) == Augmentation == * When two or more skills are applicable to a certain task (e.g. Pilot and Computer when trying to override a sabotaged autopilot), the secondary skill can give a bonus of one-third its value (round down) to the main skill value. :(Inspired by [[:Category:HeroQuest|HeroQuest]]) == Close Combat == First is the single erratum from the published rulebooks: * Unarmed comabat strikes have a damage rating of Strength + 1 Second is the house rule. * When making an offensive action (Strike or Throw), but before dice are rolled, an attacker can nominate to forgo an attempt to cause damage, but instead acquire a combat advantage. In this case, the attacker's net Action Value (attacker's AV - defender's AV) is applied as a bonus to the attacker's next action. Note that the net AV can be negative, giving the attacker a penalty. Also note that the combat advantage resets to zero if it is not used in the attacker's next action. == Equipment == * The electronic weapon scope does not give a bonus to all shots; instead, it alleviates any range penalites by 2. Its range-trebling effect remains unchanged. * The targeting interlink does not interface with programmed reflexes to provide a 'never miss' effect. It still gives a +2/+3 bonus to all shots when used. == Biomods == * The diver aquaform includes the improved blood oxygenation modification. Characters with either modification have the union of the stated benefits: they can hold their breath for up to 10 minutes while performing strenuous activity, and up to an hour if resting. * Multiglands can produce a number of effects. See [[what multiglands can do for you]] for details. [[Category:Blue Planet]] a896125c261b05ddd2a38e51dd9a5a80ad6a46fb 14 1514 1683 2008-05-16T19:03:58Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. You'll probably want to read the [[server setup|instructions in sequence]] so that they make sense. fb058ecf5ccd7c9cd7ff809aa30f4710eabe5c83 0 1 1684 1682 2008-05-16T19:04:17Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference). == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 820478e99d09c3f51d0ad236bc057d3cb2d88641 0 1515 1685 2008-05-16T19:29:21Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS Installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS master server]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS Installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS slave server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} [[Category:server setup]] 9bcb2696ae3d9677a829fe6589c8b2fa000f9d04 1686 1685 2008-05-16T19:31:16Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running Ubuntu Linux. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS Installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS master server]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS Installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS slave server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} [[Category:server setup]] 1baa71be53c1709b5db593b4a6796a0a60aeae20 1687 1686 2008-05-16T19:31:59Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS Installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS master server]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS Installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS slave server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} [[Category:server setup]] 90472ddcea5276a1b4bc8fca288a68d0e9cc4753 1693 1687 2008-05-18T19:49:21Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS Installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS master server]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS Installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS slave server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: # sudo -i and give your password. [[Category:server setup]] ad6dec19ec58868e6499865eea11d017b1a7f998 1698 1693 2008-05-18T23:04:29Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS master server]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server|DNS slave server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: # sudo -i and give your password. [[Category:server setup]] 33ce304079205d9c84d871a1b06fbd5ed556571c 1703 1698 2008-05-19T10:09:04Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: # sudo -i and give your password. [[Category:server setup]] b62f74ccf5fdf8a896c5de165757d8434432f754 1706 1703 2008-05-19T13:17:46Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server@~# sudo -i and give your password. [[Category:server setup]] ee32581ce88d19fe857351f1aa5f16d212159ac9 1709 1706 2008-05-19T13:25:38Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] 7a64f4151f814cafd117a3f4617dca18eec8ba53 1736 1709 2008-06-09T19:51:58Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] 0af73cc50a43a44a6d3b3f4bef5d78131d6f61a6 1746 1736 2008-07-09T10:45:11Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] f5bdc35fbc69a48db251685dea823835d41897b3 1747 1746 2008-07-09T10:46:44Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 7.10, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] 835c7ec5d865ae8003d24d3052a61aba6c5ec431 1754 1747 2008-07-11T06:53:33Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] 0dc4f2234915206d203f16e6f83f857605c188d9 1759 1754 2008-07-17T21:15:22Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * ''[[Ruby on Rails setup]]'' | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] fb75f1fff68b8af9a42a51f541d4c4702eaf4385 0 1516 1688 2008-05-16T19:45:12Z Neil 2 wikitext text/x-wiki Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/products/WhatIsUbuntu/serveredition Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a couple of GB each for <tt>/var</tt> and <tt>/tmp</tt>, and everything else just goes into one large partition. The server needs the additional packages <tt>build-essentials</tt> (for basic software development) and <tt>openssh-server</tt> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. [[Category:Server setup]] aa3d30b60d43fa69d067144973248f972e0173a0 1689 1688 2008-05-16T19:48:02Z Neil 2 wikitext text/x-wiki Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/products/WhatIsUbuntu/serveredition Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a couple of GB each for <tt>/var</tt> and <tt>/tmp</tt>, and everything else just goes into one large partition. The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <tt>build-essentials</tt> (for basic software development) and <tt>openssh-server</tt> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. [[Category:Server setup]] 3413fdc54098ba0e68cadb577ced2827f74b7586 1695 1689 2008-05-18T23:03:56Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em;background-color:#dfdfdf;" |colspan="2" align="center"|&uarr; [[Server setup]] |- | &larr; Previous |align="right"| Next &rarr; |- | None |align="right"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/products/WhatIsUbuntu/serveredition Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a couple of GB each for <tt>/var</tt> and <tt>/tmp</tt>, and everything else just goes into one large partition. The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <tt>build-essentials</tt> (for basic software development) and <tt>openssh-server</tt> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. [[Category:Server setup]] aa09204a5bdc4a5dfd22fd0451ad09dbd6d3be25 1696 1695 2008-05-18T23:04:10Z Neil 2 [[OS Installation]] moved to [[OS installation]] wikitext text/x-wiki {|align="right" style="width:22em;background-color:#dfdfdf;" |colspan="2" align="center"|&uarr; [[Server setup]] |- | &larr; Previous |align="right"| Next &rarr; |- | None |align="right"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/products/WhatIsUbuntu/serveredition Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a couple of GB each for <tt>/var</tt> and <tt>/tmp</tt>, and everything else just goes into one large partition. The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <tt>build-essentials</tt> (for basic software development) and <tt>openssh-server</tt> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. [[Category:Server setup]] aa09204a5bdc4a5dfd22fd0451ad09dbd6d3be25 1712 1696 2008-05-21T11:26:49Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| None |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/products/WhatIsUbuntu/serveredition Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a couple of GB each for <tt>/var</tt> and <tt>/tmp</tt>, and everything else just goes into one large partition. The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <tt>build-essentials</tt> (for basic software development) and <tt>openssh-server</tt> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. [[Category:Server setup]] be3acfb45d29a7bf2761689bacf798243f606d95 0 1526 1748 2008-07-09T10:51:36Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='la -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) [[Category:Server setup]] dde3bbd74769f0713b4b63dae02b7673e88670d8 1749 1748 2008-07-09T13:37:37Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='la -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) [[Category:Server setup]] e0025016050523fc475e26ac23fd81068306baf9 0 1530 1760 2008-07-17T21:17:12Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[DNS server setup|DNS]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <tt>/etc/ssl/domain</tt>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create the <tt>/etc/ssl/openssl.cnf</tt> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME = . RANDFILE = $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids # To use this configuration file with the "-extfile" option of the # "openssl x509" utility, name here the section containing the # X.509v3 extensions to use: # extensions = # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca' and 'req'. # Add a simple OID like this: # testoid1=1.2.3.4 # Or use config file substitution like this: # testoid2=${testoid1}.5.6 #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. #unique_subject = no # Set to 'no' to allow creation of # several ctificates with same subject. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/crl.pem # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert # Comment out the following two lines for the "traditional" # (and highly broken) format. name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options # Extension copying option: use with caution. # copy_extensions = copy # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs # so this is commented out by default to leave a V1 CRL. # crlnumber must also be commented out to leave a V1 CRL. # crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering # A few difference way of specifying how similar the request should look # For type CA, the listed attributes must be the same, and the optional # and supplied fields are just that :-) policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy # At this point in time, you must list all acceptable 'object' # types. [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert # Passwords for private keys if not present they will be prompted for # input_password = secret # output_password = secret # This sets a mask for permitted string types. There are several options. # default: PrintableString, T61String, BMPString. # pkix : PrintableString, BMPString. # utf8only: only UTF8Strings. # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). # MASK:XXXX a literal mask value. # WARNING: current versions of Netscape crash on BMPStrings or UTF8Strings # so use this option with caution! string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name # we can do this but it is not needed normally :-) #1.organizationName = Second Organization Name (eg, company) #1.organizationName_default = World Wide Web Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) #organizationalUnitName_default = commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 # SET-ex3 = SET extension number 3 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName [ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] # Extensions for a typical CA # PKIX recommendation. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always # This is what PKIX recommends but some broken software chokes on critical # extensions. #basicConstraints = critical,CA:true # So we do this instead. basicConstraints = CA:true # Key usage: this is typical for a CA certificate. However since it will # prevent it being used as an test self-signed certificate it is best # left out by default. # keyUsage = cRLSign, keyCertSign # Some might want this also # nsCertType = sslCA, emailCA # Include email address in subject alt name: another PKIX recommendation # subjectAltName=email:copy # Copy issuer details # issuerAltName=issuer:copy # DER hex encoding of an extension: beware experts only! # obj=DER:02:03 # Where 'obj' is a standard or added object # You can even override a supported extension: # basicConstraints= critical, DER:30:03:01:01:FF [ crl_ext ] # CRL extensions. # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. # issuerAltName=issuer:copy authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] # These extensions should be added when creating a proxy certificate # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName # This really needs to be in place for it to be a proxy certificate. proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] c8b542fe3a3b34020d93bacf50bcb8ea75edff7a 0 1531 1761 2008-07-17T21:18:23Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki.dovecot.org Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki.dovecot.org/LDA Deliver] local delivery agent, to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Postfix, TLS, and the Amavis + SpamAssassin + ClamAV are here. Dovecot, Deliver, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Basic Postfix with virtual users == Here we get Postfix running, by sending mail to other sites, and receiving mail and putting it in virtual users' mailboxes. I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /home/vmail/domain1.com/user1/ /user2/ /domain2.com/user1/ /user2/ /domain3.com/user1/ ...and so on. The <tt>vmail</tt> user will be the one used for email transport into and out of these virtual mailboxes. * Install postfix root@server:~# apt-get install postfix mailx * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update <tt>/etc/postfix/main.cf</tt> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <tt>/etc/postfix/vhosts</tt> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <tt>/etc/postfix/vmaps</tt> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <tt>/etc/postfix/vmaps</tt> is changed) * Create aliases for the users in <tt>/etc/postfix/valiases</tt> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <tt>/etc/postfix/valiases</tt> is changed) * Create <tt>/etc/aliases</tt> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <tt>telnet</tt> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <tt>mail</tt> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <tt>/etc/postfix/main.cf</tt> file to point to the certificates # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <tt>quit</tt> to end the session. SASL is mentioned in various places, but it isn't needed. SASL allows access to the SMTP server from remote sites. This Postfix installation is configured so that only machines on my LAN can use it. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <tt>/etc/mailname</tt> mydomain.com * Install the packages: root@server:~# apt-get install amavis-new spamassassin clamav === ClamAV === * Add the ClamAV user to the Amavis group root@server:~# adduser clamav amavis The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <tt>/etc/default/spamassassin</tt> to activate the Spamassassin daemon. Change <tt>ENABLED=0</tt> to: ENABLED=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <tt>/etc/amavis/conf.d/15-content_filter_mode</tt> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <tt>/etc/amavis/conf.d/20-debian-defaults</tt> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <tt>/etc/postfix/main.cf</tt> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <tt>/etc/postfix/master.cf</tt> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <tt>recipient_delimiter</tt> option on Postfix is rather neat. Given a <tt>recipient_delimiter</tt> of '<tt>.</tt>', messages to <tt>user.ext1@domain1.com</tt> are handled in an interesting way. If <tt>user.ext1@domain1.com</tt> is a valid user, the message is put in their mailbox. If <tt>user.ext1@domain1.com</tt> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <tt>user@domain1.com</tt>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <tt>/etc/postfix/main.cf</tt> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <tt>/etc/postfix/header_checks</tt> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <tt>/etc/postfix/main.cf</tt> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com maximal_queue_lifetime = 21d * Reload the configuration root@server:~# postfix reload That handles getting mail into the server, and into other servers. The next page deals with getting them out again, using [[Dovecot server setup|Dovecot as a LDA]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 8af5dc05a7b25417782f1d8a3d507acdbb69e246 0 1532 1762 2008-07-17T21:19:41Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This is all about the user-facing side of setting up a mail server. For information on setting up the server-facing side, see the [[Postfix server setup]] page. Also see the [[Webmail setup]] page. The basic source for setting up Postfix and Dovecot is the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois :(<tt>whois</tt> is needed in a moment for creating passwords) * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain digest-md5 userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <tt>/etc/dovecot/dovecot.conf</tt> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = cmusieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <tt>/etc/postfix/main.cf</tt> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <tt>/etc/postfix/master.cf</tt> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <tt>/home/vmail/domain1.com/user1/.dovecot.sieve</tt> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <tt>.dovecot.sieve</tt> file. The first time Dovecot delivers mail to this user, it will create the <tt>.dovecot.sievec</tt> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 42f5a981c911a7ef85ff3369343319e01bbe6607 1777 1762 2008-07-18T10:48:59Z Neil 2 /* Set up local delivery and sieve */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This is all about the user-facing side of setting up a mail server. For information on setting up the server-facing side, see the [[Postfix server setup]] page. Also see the [[Webmail setup]] page. The basic source for setting up Postfix and Dovecot is the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois :(<tt>whois</tt> is needed in a moment for creating passwords) * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain digest-md5 userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <tt>/etc/dovecot/dovecot.conf</tt> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = cmusieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <tt>/etc/postfix/main.cf</tt> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <tt>/etc/postfix/master.cf</tt> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Create an an empty log file for <tt>deliver</tt> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <tt>/home/vmail/domain1.com/user1/.dovecot.sieve</tt> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <tt>.dovecot.sieve</tt> file. The first time Dovecot delivers mail to this user, it will create the <tt>.dovecot.sievec</tt> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] b0da410ffabd26a81523ed520fcef14588b4dab6 0 1533 1763 2008-07-17T21:20:43Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <tt>/etc/apache2/sites-available</tt>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <tt>ServerName</tt> and <tt>DocumentRoot</tt> settings to reflect the site's name and location of files. Also note the non-default location of the <tt>cgi-bin</tt> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <tt>root</tt>, and permissions of 755. Also note that the contents of the <tt>/usr/share/doc/</tt> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 restart == Secure HTTP == I use the Secure HTTP server for my webmail. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <tt>/etc/apache2/ports.conf</tt> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <tt><nowiki>www.domain.tld</nowiki></tt> == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] [[Category:Server setup]] b0b88ae5f12bf7775d3e9975b60d02715b562b54 0 1534 1764 2008-07-17T21:22:11Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp /usr/share/mediawiki/LocalSettings.php . root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/upload . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data upload root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# /etc/init.d/apache2 reload * Now got to <tt><nowiki>http://site.domain.tld/mediawiki/</nowiki></tt> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <tt>config/LocalSettings.php</tt> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <tt>$IP</tt> at the top of the file. * With the wiki set up, remove the <tt>config</tt> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <tt>LocalSettings.php</tt> file. The changes are marked below with <tt>###</tt> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <tt>/mediawiki/</tt> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <tt>/etc/apache2/sites-available/site.domain.tld</tt> file, in the place specified above. RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <tt><nowiki>http://site.domain.tld/Main_Page</nowiki></tt> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <tt>LocalSettings.php</tt>, modify the <tt>$wgEnableUploads</tt> setting: $wgEnableUploads = true; === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <tt>LocalSettings.php</tt> file as shown. Add the lines just below the <tt>$wgEnableUploads</tt> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <tt>mediawiki-math</tt> pagkage is installed, it can be enabled with a simple edit of <tt>LocalSettings.php</tt>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <tt>update.php</tt> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <tt>AdminSettings.php</tt> file. Instead, do the following: * Take a backup of the wiki database * Set up the new wiki, following the instructions as above, but without deleting the <tt>config</tt> directory * Overwrite the wiki database with the backed-up version * Remove the <tt>LocalSettings.php</tt> file * Go back to the wiki's config page. This will update the database schema * Once that's done, copy the new <tt>LocalSettings.php</tt> into the Mediawiki root and continue with the configuration. == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) [[Category:Server setup]] b1f13304d8d027d59614cad1178fadd8a0c8a69d 1770 1764 2008-07-18T07:12:39Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/upload . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data upload root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# /etc/init.d/apache2 reload * Now got to <tt><nowiki>http://site.domain.tld/mediawiki/</nowiki></tt> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <tt>config/LocalSettings.php</tt> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <tt>$IP</tt> at the top of the file. * With the wiki set up, remove the <tt>config</tt> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <tt>LocalSettings.php</tt> file. The changes are marked below with <tt>###</tt> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <tt>/mediawiki/</tt> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <tt>/etc/apache2/sites-available/site.domain.tld</tt> file, in the place specified above. RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <tt><nowiki>http://site.domain.tld/Main_Page</nowiki></tt> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <tt>LocalSettings.php</tt>, modify the <tt>$wgEnableUploads</tt> setting: $wgEnableUploads = true; === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <tt>LocalSettings.php</tt> file as shown. Add the lines just below the <tt>$wgEnableUploads</tt> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <tt>mediawiki-math</tt> pagkage is installed, it can be enabled with a simple edit of <tt>LocalSettings.php</tt>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <tt>update.php</tt> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <tt>AdminSettings.php</tt> file. Instead, do the following: * Take a backup of the wiki database * Set up the new wiki, following the instructions as above, but without deleting the <tt>config</tt> directory * Overwrite the wiki database with the backed-up version * Remove the <tt>LocalSettings.php</tt> file * Go back to the wiki's config page. This will update the database schema * Once that's done, copy the new <tt>LocalSettings.php</tt> into the Mediawiki root and continue with the configuration. == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) [[Category:Server setup]] 0f0713820374c96e0b5d58586580423b96efa548 1771 1770 2008-07-18T07:54:31Z Neil 2 /* Rewriting paths */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/upload . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data upload root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# /etc/init.d/apache2 reload * Now got to <tt><nowiki>http://site.domain.tld/mediawiki/</nowiki></tt> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <tt>config/LocalSettings.php</tt> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <tt>$IP</tt> at the top of the file. * With the wiki set up, remove the <tt>config</tt> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <tt>LocalSettings.php</tt> file. The changes are marked below with <tt>###</tt> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <tt>/mediawiki/</tt> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <tt>/etc/apache2/sites-available/site.domain.tld</tt> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <tt><nowiki>http://site.domain.tld/Main_Page</nowiki></tt> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <tt>LocalSettings.php</tt>, modify the <tt>$wgEnableUploads</tt> setting: $wgEnableUploads = true; === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <tt>LocalSettings.php</tt> file as shown. Add the lines just below the <tt>$wgEnableUploads</tt> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <tt>mediawiki-math</tt> pagkage is installed, it can be enabled with a simple edit of <tt>LocalSettings.php</tt>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <tt>update.php</tt> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <tt>AdminSettings.php</tt> file. Instead, do the following: * Take a backup of the wiki database * Set up the new wiki, following the instructions as above, but without deleting the <tt>config</tt> directory * Overwrite the wiki database with the backed-up version * Remove the <tt>LocalSettings.php</tt> file * Go back to the wiki's config page. This will update the database schema * Once that's done, copy the new <tt>LocalSettings.php</tt> into the Mediawiki root and continue with the configuration. == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) [[Category:Server setup]] b803d063b8a122d9d1e88b1a9f3284f199063848 1778 1771 2008-07-19T03:00:27Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/upload . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data upload root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# /etc/init.d/apache2 reload * Now got to <tt><nowiki>http://site.domain.tld/mediawiki/</nowiki></tt> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <tt>config/LocalSettings.php</tt> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <tt>$IP</tt> at the top of the file. * With the wiki set up, remove the <tt>config</tt> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <tt>LocalSettings.php</tt> file. The changes are marked below with <tt>###</tt> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <tt>/mediawiki/</tt> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <tt>/etc/apache2/sites-available/site.domain.tld</tt> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <tt><nowiki>http://site.domain.tld/Main_Page</nowiki></tt> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <tt>LocalSettings.php</tt>, modify the <tt>$wgEnableUploads</tt> setting: $wgEnableUploads = true; === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <tt>LocalSettings.php</tt> file as shown. Add the lines just below the <tt>$wgEnableUploads</tt> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <tt>mediawiki-math</tt> pagkage is installed, it can be enabled with a simple edit of <tt>LocalSettings.php</tt>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <tt>update.php</tt> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <tt>AdminSettings.php</tt> file. Instead, do the following: * Take a backup of the wiki database * Set up the new wiki, following the instructions as above, but without deleting the <tt>config</tt> directory * Overwrite the wiki database with the backed-up version * Remove the <tt>LocalSettings.php</tt> file * Go back to the wiki's config page. This will update the database schema * Once that's done, copy the new <tt>LocalSettings.php</tt> into the Mediawiki root and continue with the configuration. == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 414e00610de1d945da09eef091b512809b1354b1 1792 1778 2008-07-21T09:49:52Z Neil 2 /* Enabling uploads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/upload . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data upload root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# /etc/init.d/apache2 reload * Now got to <tt><nowiki>http://site.domain.tld/mediawiki/</nowiki></tt> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <tt>config/LocalSettings.php</tt> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <tt>$IP</tt> at the top of the file. * With the wiki set up, remove the <tt>config</tt> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <tt>LocalSettings.php</tt> file. The changes are marked below with <tt>###</tt> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <tt>/mediawiki/</tt> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <tt>/etc/apache2/sites-available/site.domain.tld</tt> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <tt><nowiki>http://site.domain.tld/Main_Page</nowiki></tt> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <tt>LocalSettings.php</tt>, modify the <tt>$wgEnableUploads</tt> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <tt>LocalSettings.php</tt> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <tt>LocalSettings.php</tt> file as shown. Add the lines just below the <tt>$wgEnableUploads</tt> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <tt>mediawiki-math</tt> pagkage is installed, it can be enabled with a simple edit of <tt>LocalSettings.php</tt>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <tt>update.php</tt> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <tt>AdminSettings.php</tt> file. Instead, do the following: * Take a backup of the wiki database * Set up the new wiki, following the instructions as above, but without deleting the <tt>config</tt> directory * Overwrite the wiki database with the backed-up version * Remove the <tt>LocalSettings.php</tt> file * Go back to the wiki's config page. This will update the database schema * Once that's done, copy the new <tt>LocalSettings.php</tt> into the Mediawiki root and continue with the configuration. == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 7bf52f0247b348b513de16d82cf63297ed15353c 0 1535 1765 2008-07-17T21:22:37Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} This one's easy. I use [http://www.squirrelmail.org/index.php SquirrelMail] as a webmail interface. To install Squirrelmail, install the packages: root@server:~# apt-get install squirrelmail ibritish (which also includes a British English dictionary). SquirrelMail has a very nice configuration script that does just about all the work for you, given that the [[Dovecot server setup|IMAP server]] and [[Web server setup|Web server]] are already working. Simply run the script: root@server:~# squirrelmail-configure select 'D' to configure using Dovecot, and follow the instructions. As SquirrelMail, Apache, and Dovecot are all running on the same machine, there's no need to use IMAPS to secure the communication between them. Next, you need to modify the <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> file to point to SquirrelMail. Adjust the <tt>DocumentRoot</tt> setting in the secure section of that file to DocumentRoot /usr/share/squirrelmail Restart the server: root@server:~# /etc/init.d/apache2 restart And you should have a SquirrelMail webmail site running! If you want to change the logo on the login screen, replace the file <tt>/usr/share/squirrelmail/images/sm_logo.png</tt> with another image, making sure it's the same width and height as the original. == See also == * [http://www.squirrelmail.org/documentation/ SquirrelMail documentation] * [https://help.ubuntu.com/community/Squirrelmail Hints for SquirrelMail on Ubuntu] [[Category:Server setup]] 2f4459b50a3f9679ea57914da6d604593138f7c4 1769 1765 2008-07-17T21:35:22Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} This one's easy. I use [http://www.squirrelmail.org/index.php SquirrelMail] as a webmail interface. To install Squirrelmail, install the packages: root@server:~# apt-get install squirrelmail ibritish (which also includes a British English dictionary). SquirrelMail has a very nice configuration script that does just about all the work for you, given that the [[Dovecot server setup|IMAP server]] and [[Web server setup|Web server]] are already working. Simply run the script: root@server:~# squirrelmail-configure select 'D' to configure using Dovecot, and follow the instructions. As SquirrelMail, Apache, and Dovecot are all running on the same machine, there's no need to use IMAPS to secure the communication between them. Next, you need to modify the <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> file to point to SquirrelMail. Adjust the <tt>DocumentRoot</tt> setting in the secure section of that file to DocumentRoot /usr/share/squirrelmail Restart the server: root@server:~# /etc/init.d/apache2 restart And you should have a SquirrelMail webmail site running! If you want to change the logo on the login screen, replace the file <tt>/usr/share/squirrelmail/images/sm_logo.png</tt> with another image, making sure it's the same width and height as the original. To migrate from a previous installation, copy the files in <tt>/var/lib/squirrelmail/data</tt> to keep users' preferences. == See also == * [http://www.squirrelmail.org/documentation/ SquirrelMail documentation] * [https://help.ubuntu.com/community/Squirrelmail Hints for SquirrelMail on Ubuntu] [[Category:Server setup]] f3492d1f803856f63b6021c00392731766fec578 0 1536 1766 2008-07-17T21:22:57Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Backup setup|Backup]] |} == See also == Here are a few pages that are useful guides or provide background and context. [[Category:Server setup]] 50582a95e98edf3f00c0aef066582e670d255699 0 1538 1768 2008-07-17T21:23:33Z Neil 2 New page: Install the following: apt-get install libapache2-mod-ruby ruby ruby-elisp rubygems wikitext text/x-wiki Install the following: apt-get install libapache2-mod-ruby ruby ruby-elisp rubygems 2086cfb1f1137f4e93305526038e7209642168a7 0 1540 1776 2008-07-18T09:44:50Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Desktop backup == On the desktop machine, there is <tt>/etc/cron.daily/makedailybackups</tt>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +1 -exec rm -f {} \; and <tt>/etc/cron.monthly/makemonthlybackups</tt> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -ctime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <tt>--absolute-names</tt> would restore the files to paths anchored at root. == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <tt>/etc/cron.daily/makedailybackups</tt>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +2 -exec rm -f {} \; # Create file backups tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do mysqldump --user='root' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null [[Category:Server setup]] 7b00e4f8de3cbc3985d574cef7b3a9f98d577aa0 1793 1776 2008-07-21T11:24:04Z Neil 2 /* Server backup */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Desktop backup == On the desktop machine, there is <tt>/etc/cron.daily/makedailybackups</tt>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +1 -exec rm -f {} \; and <tt>/etc/cron.monthly/makemonthlybackups</tt> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -ctime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <tt>--absolute-names</tt> would restore the files to paths anchored at root. == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <tt>/etc/cron.daily/makedailybackups</tt>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +2 -exec rm -f {} \; # Create file backups tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null [[Category:Server setup]] 695b9dce0970b6d3b7b39fc40031536c9ad9c26e 0 1541 1779 2008-07-19T03:19:57Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <tt>/etc/hostname</tt> to server.domain.tld == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 * Change <tt>/etc/resolv.conf</tt> to nameserver 192.168.1.1 (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. [[Category:Server setup]] bf8f4040731bf6d1c724148cbc80acef1d6bb0a8 1785 1779 2008-07-21T09:04:46Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <tt>/etc/hostname</tt> to server.domain.tld == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 * Change <tt>/etc/resolv.conf</tt> to nameserver 192.168.1.1 (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Regenerating OpenSSH keys == If you change the hostname after setting up the machine, the OpenSSH server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) [[Category:Server setup]] 3bee51b4fe8ae9f09ba24433c492bee1e98aceca 0 1542 1780 2008-07-19T03:26:02Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <tt>localhost</tt>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <tt>/etc/cups/cupsd.conf</tt> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock then restart CUPS: root@server:~# /etc/init.d/cupsys restart You should then be able to point a webbrowser at <tt>desktop.server.tld:631</tt> or <tt>server.domain.tld:631</tt> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <tt>Administration</tt> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <tt><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></tt>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. [[Category:Server setup]] d4b2be2a36560fde163f92e5efc7d273e99c3584 1781 1780 2008-07-19T03:32:02Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <tt>localhost</tt>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <tt>/etc/cups/cupsd.conf</tt> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock and add the <tt>Allow @LOCAL</tt> lines in the two <tt>Location</tt> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# /etc/init.d/cupsys restart You should then be able to point a webbrowser at <tt>desktop.server.tld:631</tt> or <tt>server.domain.tld:631</tt> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <tt>Administration</tt> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <tt><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></tt>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. [[Category:Server setup]] fe23d8436140e1dc3288edd98a3cfe9f8fcdbe83 0 1543 1782 2008-07-19T03:33:58Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <tt>/proc</tt> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. * Add the <tt>/etc/iptables.rules</tt> file and the <tt>/etc/iptables</tt> file. Adjust <tt>/etc/iptables.rules</tt> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> === Iptables.rules === This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END === Iptables script === #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] f985fe8823915b57b3acde19212368ba276ba515 0 1544 1783 2008-07-19T03:37:30Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == References == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers forwarders { 1.2.3.4; 1.2.3.5; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld NS desktop.domain.tld ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld : (and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/bind/run root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run * Modify the file <tt>/etc/default/bind9</tt> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * Modify <tt>/etc/default/syslogd</tt> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * Make AppArmor allow BIND access to the chroot jail. Modify <tt>/etc/apparmor.d/usr.sbin.named</tt> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/run/bind/run/named.pid w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, Sysklogd, and BIND: root@server:~# /etc/init.d/apparmor restart root@server:~# /etc/init.d/sysklogd restart root@server:~# /etc/init.d/bind9 restart * Check <tt>/var/log/syslog</tt> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. [[Category:Server setup]] 95c13ea9a2f74c131d9cdfbc0b697cab9fb0649b 0 1545 1784 2008-07-19T03:40:11Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Web server setup|Web server]] |} The only thing to do here is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords. The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <tt>hostname</tt> is the name of this host, e.g. <tt>server.domain.tld</tt>. * Add passwords to the <tt>root</tt> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); And that's it! == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] a8cde8cb3dc924d99653eb653098a806b629d319 6 1546 1786 2008-07-21T09:34:04Z Neil 2 Expanded table of contents of the Blue Planet Moderator's Guide. wikitext text/x-wiki Expanded table of contents of the Blue Planet Moderator's Guide. a3c1cafc9731b2035847fd1cb68e4c4b02ec4aed 1789 1786 2008-07-21T09:41:27Z Neil 2 wikitext text/x-wiki Expanded table of contents of the [[:Category:Blue Planet|Blue Planet]] [[Blue_Planet_Moderators_Guide_contents|Moderator's Guide]]. [[Category:Blue Planet]] 78cd07c71f6fda66e220194bf2cda30233fbb9b1 6 1547 1787 2008-07-21T09:34:50Z Neil 2 Expanded table of contents of the Blue Planet Player's Guide. wikitext text/x-wiki Expanded table of contents of the Blue Planet Player's Guide. 7ab6c392907f9c9503974cd2fa257b8f425ea1dc 1788 1787 2008-07-21T09:37:51Z Neil 2 wikitext text/x-wiki Expanded table of contents of the [[:Category:Blue Planet|Blue Planet]] [[Blue_Planet_Players_Guide_contents|Player's Guide]]. [[Category:Blue Planet]] 93c449a5112663abfb385cc816b53a8d772ba479 0 1476 1790 1614 2008-07-21T09:42:59Z Neil 2 wikitext text/x-wiki This is an extended table of contents for the [[:Category:Blue Planet|Blue Planet]] Moderators' Guide. You can also [[:Image:Moderators-guide-contents.rtf.gz|download this as a compressed .rtf file]]. {| |- | '''''Foreword: The Moderator's Role''''' || '''''3''''' |- | &nbsp;&nbsp;'''Easter''' || '''3''' |- | &nbsp;&nbsp;'''The Moderator's Role''' || '''4''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Designing Campaigns'' || ''4'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Campaign Concept || 4 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Scenario Ideas || 6 |- | &nbsp;&nbsp;'''The Moderator's Objective''' || '''7''' |- | '''''Chapter 1: The New Frontier''''' || '''''8''''' |- | &nbsp;&nbsp;'''The Serpentis System''' || '''9''' |- | &nbsp;&nbsp;'''The Pacifica Archipelago''' || '''13''' |- | &nbsp;&nbsp;'''Haven Cluster''' || '''15''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Haven'' || ''18'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 18 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 18 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 19 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Districts of Haven || 19 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Jason and Darwin || 20 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 22 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 22 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 23 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 24 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 24 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 25 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Second Try'' || ''26'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 26 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 26 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 28 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Districts of Second Try || 29 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 30 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 30 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 30 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 30 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 31 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 32 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Lebensraum'' || ''32'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 32 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 33 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 33 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 33 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Nomad'' || ''35'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 35 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 35 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 36 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 36 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Depth Perception || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 37 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Circumstance'' || ''38'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 38 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 38 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 39 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 39 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 39 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 40 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Wall'' || ''40'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''42'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Milo Franklin || 42 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Vendor || 43 |- | &nbsp;&nbsp;'''New Hawaii''' || '''45''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Simushir'' || ''47'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 47 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 47 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 49 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 49 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 50 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 50 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 50 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 51 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 52 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Atlantis'' || ''52'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 52 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 52 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Underground || 54 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Coronado Station'' || ''55'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 55 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 55 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 55 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 55 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Members Only || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''56'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Brian Steeg || 56 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Rachel Lightfoot || 57 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 58 |- | &nbsp;&nbsp;'''Northwest Territories''' || '''61''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 63 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Sierra Nueva Cluster'' || ''63'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 63 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 63 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 64 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 64 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 64 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 65 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 65 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Native Unrest || 65 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 66 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Baffin Island Settlement'' || ''66'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 66 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 66 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Warpod || 68 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 70 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 71 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 73 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Santa Elena'' || ''74'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 74 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 74 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 74 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 75 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Crusoe Island Military Base'' || ''77'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 77 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 77 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 78 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 79 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Sierra Neuva War'' || ''79'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 82 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''83'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Prophet || 83 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Stuart Parsons || 83 |- | &nbsp;&nbsp;'''Prime Meridian''' || '''85''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Al-Mamlakah'' || ''87'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 89 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 89 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 90 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 90 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 91 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 92 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Undersea Habitat-2'' || ''93'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 93 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 95 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Under Pressure || 96 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 96 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Alderberg'' || ''97'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 97 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 97 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 97 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 97 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 98 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 98 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 98 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 98 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Kansas'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 99 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 99 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 99 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 100 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 100 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 100 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 100 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 101 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Islamic Faithful'' || ''101'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Meridian Frontier'' || ''102'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Bryon's Spur || 102 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Maglev Express || 102 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Stockyards || 103 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 103 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''104'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Habib Hussein Al-Muhammadi || 104 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Simon Malmoneeds || 105 |- | &nbsp;&nbsp;'''Westscape''' || '''107''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dyfedd'' || ''108'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 108 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 108 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 110 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Districts of Dyfedd || 111 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 112 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 113 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 113 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 113 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hugo D-4 || 113 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 114 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 115 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Legacy of Recontact'' || ''116'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Perdition || 116 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Landgrab || 117 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Hanover Presence || 118 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Uncertain Future || 119 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Brush War || 119 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 120 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;B&ouml;se Strand || 120 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 121 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''122'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Johnny Cecily || 122 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Bernardo Oloveira || 122 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Philip Jamhuri || 123 |- | &nbsp;&nbsp;'''Zion Islands''' || '''124''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 126 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Kingston'' || ''127'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 127 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 127 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 129 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Parishes of Kingston || 129 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 131 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 131 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Map Key || 131 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 132 |- | &nbsp;&nbsp;&nbsp;&nbsp;''New Fremantle'' || ''134'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 136 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 136 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fort Pacifica'' || ''137'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 137 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 137 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 137 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 137 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 138 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 138 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 138 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 138 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Nothing Ventured'' || ''139'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Hazards Casino || 139 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dragon Boy Parlour || 140 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Kingston Saloons || 140 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Bright Savanna'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Most Wanted'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Roberto &ldquo;Sugar&rdquo; McKay || 141 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Geronimo Pacheco || 142 |- | &nbsp;&nbsp;'''A View from Orbit''' || '''143''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Prosperity Station'' || ''143'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location and Local Terrain || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Shannon Marie || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrastructure || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Space Dock || 145 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Worldscape || 145 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 145 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon's Moons'' || ''146'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Proteus || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nereus || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Serpentis Belt'' || ''146'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 147 |- | '''''Chapter 2: Beyond the Frontier''''' || '''''148''''' |- | &nbsp;&nbsp;'''Oceanography for Gamers''' || '''149''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Pressure'' || ''149'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Breathing Underwater'' || ''149'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Light Underwater'' || ''150'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sound Underwater'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Temperature'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Salinity'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Tides'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Currents'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Waves'' || ''152'' |- | &nbsp;&nbsp;'''Poseidon Ecosystems''' || '''152''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Canyonlands'' || ''152'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Mangrove'' || ''153'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sargassum Islands'' || ''154'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Thermal Oases'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Tidal Mud Reefs'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Fever || 156 |- | &nbsp;&nbsp;'''The Forecast''' || '''157''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''General Climate'' || ''157'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cyclonic Storms'' || ''158'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GEO Metwatch Force Ratings || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cyclonics in the Game || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Storm Season || 160 |- | &nbsp;&nbsp;'''Poseidon Field Guide''' || '''163''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Am-Bush'' || ''163'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Blimp'' || ''164'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Carniflora'' || ''165'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Chain Beetle'' || ''166'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Digger Crab'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Eel Dragon'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fast Fungus'' || ''168'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fish'' || ''169'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fisherman'' || ''170'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ghoster'' || ''170'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Greater White'' || ''171'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hangin' Joe'' || ''172'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Harvester Worm'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hatchlings'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hexa Boar'' || ''174'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Howell's Leech'' || ''175'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Land Lizard'' || ''175'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Loggerhead'' || ''176'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Marsh Devil'' || ''177'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Needle Bush'' || ''177'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Needle Shell'' || ''178'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Night Crawler'' || ''179'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Nooniebird'' || ''179'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 180 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Polypod'' || ''180'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Kelp'' || ''181'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Mangrove'' || ''182'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Scorpion'' || ''182'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poseidon Trilobyte'' || ''183'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Reefworm'' || ''184'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Rubber Shrimp'' || ''184'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Rumble Bee'' || ''185'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Schooler'' || ''185'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Seaweaver'' || ''186'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spurts'' || ''187'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Squealers, Stick Monkeys'' || ''188'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Stone Snake'' || ''188'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sunburst'' || ''189'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Trident Fish'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Walkabout'' || ''191'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Dart'' || ''192'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Hemp'' || ''192'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Rat'' || ''193'' |- | '''''Chapter 3: Alien Legacy''''' || '''''194''''' |- | &nbsp;&nbsp;'''The Aborigines''' || '''196''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creator Legacy'' || ''196'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aboriginal Culture'' || ''196'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Technology'' || ''196'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Long John'' || ''197'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Anatomy and Physiology'' || ''197'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Morphology and Locomotion || 197 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nervous System || 198 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Senses || 198 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Metabolism || 198 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Electrochemical Abilities || 198 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reproduction || 199 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aborigine Castes'' || 199 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Breeders || 199 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Technicians || 199 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Specialists || 200 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Herders || 200 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Protectors || 200 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aborigines and the Human Invasion'' || ''201'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creator Caches'' || ''201'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Power Sources'' || ''203'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creator Facilities'' || ''203'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Machine Farm || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Storage Cache || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Template Library || 204 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hibernation Dormitory || 204 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Research Sites || 204 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creator Technology'' || ''204'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nanites || 204 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Biomechanical Drone || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Memory Ring || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Repair Bath || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Raw Materials Depot || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Smart Tool || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Terraforming Reactor || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Xenosilicate Template || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aborigines of the Pacifica Archipelago'' || ''206'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Haven Cluster || 207 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Missing || 208 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;New Hawaii || 209 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Northwest Territories || 212 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Prime Meridian || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Westscape || 215 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Zion Islands || 217 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Bottoms Up || 218 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Creatures of Myth and Legend'' || ''220'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Illuminated Origins || 220 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;It Feeds on your Fear || 220 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Widow's Harbingers || 221 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Aborigine Graveyard || 222 |- | '''''Chapter 4: A World of Hurt''''' || '''''223''''' |- | &nbsp;&nbsp;'''The GEO''' || '''224''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Executive Council'' || ''224'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Commissioner General || 224 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Deputy Commissioner General || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;''High Commissions'' || ''226'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Human Resources || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Natural Resources || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Science and Technology || 227 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Communications || 227 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Internal Security || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State and Internal Affairs || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armed Forces || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Justice || 231 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Trade and Industry || 231 |- | &nbsp;&nbsp;&nbsp;&nbsp;''General Assembly'' || ''232'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''World Court'' || ''233'' |- | &nbsp;&nbsp;'''The Incorporated City-States''' || '''233''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Anasi Systems'' || ''233'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Atlas Materials'' || ''233'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Biogene'' || ''234'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dundalk Shipbuilding'' || ''235'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Gendiver'' || ''235'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hanover Industries'' || ''236'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hydrospan'' || ''237'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Lavender Organics'' || ''238'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''MacLeod Enforcement'' || ''238'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Nippon Industrial State'' || ''239'' |- | &nbsp;&nbsp;'''Free Zones''' || '''240''' |- | &nbsp;&nbsp;'''Earth in 2199''' || '''241''' |- | &nbsp;&nbsp;'''The Solar System''' || '''241''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Luna'' || ''241'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 241 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout, Lunar Cities || 242 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 242 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 242 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 243 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Luna and Earth || 244 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Skyhook'' || ''244'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 246 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Mars Colony'' || ''246'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 246 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 246 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 247 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 247 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 247 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Mars and Earth || 247 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Access Denied || 248 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Asteroid Belt and Beyond'' || ''249'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 249 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Physical Layout || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Demographics || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Government || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Economic Base || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Recent Developments || 250 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The New Belters || 251 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hole City'' || ''251'' |} [[Category:Blue Planet]] 0f98a3ae59c96b21da0be8da5aaa72e6b0290891 0 1475 1791 1616 2008-07-21T09:43:05Z Neil 2 wikitext text/x-wiki This is an extended table of contents for the [[:Category:Blue Planet|Blue Planet]] Players' Guide. You can also [[:Image:Players-guide-contents.rtf.gz|download this as a compressed .rtf file]]. {| |- | '''''Chapter 1: Welcome to Blue Planet''''' || '''''4''''' |- | &nbsp;&nbsp;'''2199AD''' || '''5''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Lesear Effect || 5 |- | &nbsp;&nbsp;'''Blue Planet v2''' || '''6''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''What is Blue Planet?'' || ''7'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Game Moderator'' || ''7'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Synergy Game System'' || ''8'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Dice || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dice Pools || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Target Numbers || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Task Rolls || 8 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Making the Game your Own || 8 |- | &nbsp;&nbsp;'''Poseidon: A Survival Guide''' || '''9''' |- | '''''Chapter 2: Character Creation''''' || '''''25''''' |- | &nbsp;&nbsp;'''Concept''' || '''26''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Group Concept'' || ''26'' |- | &nbsp;&nbsp;'''Power Level and Game Balance''' || '''26''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Twenty Questions || 27 |- | &nbsp;&nbsp;'''Characteristics''' || '''28''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Attributes'' || ''28'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Derived Attributes'' || ''28'' |- | &nbsp;&nbsp;'''Character Species''' || '''28''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Pure-strain'' || ''29'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: In the Service of Justice || 29 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Modified'' || ''31'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human, Genetic Redesign'' || ''31'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dolphin'' || ''32'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Killer Whale'' || ''32'' |- | &nbsp;&nbsp;'''Modifying Attributes''' || '''33''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Point System'' || ''33'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Random System'' || ''33'' |- | &nbsp;&nbsp;'''Abilities''' || '''34''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Human Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Modi Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Genie Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cetacean Abilities'' || ''34'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Echolocation || 34 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Environmental Sensitivity || 34 |- | &nbsp;&nbsp;'''Aptitudes''' || '''37''' |- | &nbsp;&nbsp;'''Skills''' || '''38''' |- | &nbsp;&nbsp;'''Training Packages''' || '''39''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Custom Skills'' || ''39'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Origin Packages'' || ''40'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Background Packages'' || ''41'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Professional Packages'' || ''43'' |- | &nbsp;&nbsp;'''Skill Descriptions''' || '''51''' |- | &nbsp;&nbsp;'''Sample Character: Bush Pilot''' || '''58''' |- | &nbsp;&nbsp;'''Sample Character: Dolphin Engineer''' || '''60''' |- | &nbsp;&nbsp;'''Sample Character: Field Biologist''' || '''62''' |- | &nbsp;&nbsp;'''Sample Character: Gangster''' || '''64''' |- | &nbsp;&nbsp;'''Sample Character: GEO Marshal''' || '''66''' |- | &nbsp;&nbsp;'''Sample Character: GEO Shock Trooper''' || '''68''' |- | &nbsp;&nbsp;'''Sample Character: Native Ecoterrorist''' || '''70''' |- | &nbsp;&nbsp;'''Sample Character: Orca Peacekeeper''' || '''72''' |- | &nbsp;&nbsp;'''Sample Character: Silva Warden''' || '''74''' |- | &nbsp;&nbsp;'''Character Development''' || '''76''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Earning Character Improvement Points'' || ''76'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spending Character Improvement Points'' || ''76'' |- | '''''Chapter 3: Character Profile''''' || '''''77''''' |- | &nbsp;&nbsp;'''Goal''' || '''78''' |- | &nbsp;&nbsp;'''Motivation''' || '''79''' |- | &nbsp;&nbsp;'''Attitude''' || '''80''' |- | &nbsp;&nbsp;'''Player's Choice''' || '''80''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Features'' || ''81'' |- | &nbsp;&nbsp;'''Roles''' || '''82''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Administrator'' || ''82'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artist'' || ''83'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Civilian Pilot'' || ''83'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Doctor'' || ''84'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ecoterrorist'' || ''85'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Freebooter'' || ''85'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Frontiersman'' || ''86'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Gangster'' || ''86'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Cold Soup || 87 |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Magistrate'' || ''88'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Marshall'' || ''89'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Patrol'' || ''90'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Peacekeeper'' || ''91'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''GEO Shock Trooper'' || ''91'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Incorporate Security'' || ''92'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Intelligence Agent'' || ''92'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Journalist'' || ''93'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Medic'' || ''93'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Mercenary'' || ''94'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Military Pilot'' || ''95'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Miner'' || ''96'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Healer'' || ''97'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Insurgent'' || ''98'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Patrol'' || ''98'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Native Sell-Out'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Opportunist'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Pioneer'' || ''99'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Private Investigator'' || ''100'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Prospector'' || ''100'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sailor'' || ''101'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Black Charlie || 102 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Scholar'' || ''102'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Scientist'' || ''103'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spacer'' || ''104'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Technician'' || ''104'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Thug'' || ''106'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Trader'' || ''106'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Warden'' || ''107'' |- | '''''Chapter 4: Synergy Game System''''' || '''''108''''' |- | &nbsp;&nbsp;'''Task Resolution''' || '''109''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Skilled Tasks'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aptitude'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Target Number'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Task Difficulty'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Unskilled Tasks'' || ''109'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Action Value'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Attribute Rolls'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Opposed Rolls'' || ''110'' |- | &nbsp;&nbsp;'''Combat''' || '''110''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Action Rounds'' || ''110'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Actions'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Initiative'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Multiple Actions'' || ''111'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Holding Actions'' || ''112'' |- | &nbsp;&nbsp;'''Ranged Combat''' || '''112''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Range'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Size'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Movement'' || ''112'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cover'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aiming'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Recoil'' || ''113'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Automatic Fire'' || ''113'' |- | &nbsp;&nbsp;'''Close Combat''' || '''114''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Manoeuvres'' || ''114'' |- | &nbsp;&nbsp;'''Situational Modifiers''' || '''116''' |- | &nbsp;&nbsp;'''Damage''' || '''116''' |- | &nbsp;&nbsp;'''Wounds''' || '''117''' |- | &nbsp;&nbsp;'''Damage Resolution''' || '''117''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Called Shots'' || ''118'' |- | &nbsp;&nbsp;'''First Aid''' || '''118''' |- | &nbsp;&nbsp;'''Special Damage''' || '''118''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Falling'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Explosives'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Burning and Electrical Shock'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hypothermia'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Poison'' || ''119'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Suffocation and Drowning'' || ''120'' |- | &nbsp;&nbsp;'''Healing and Recovery''' || '''120''' |- | &nbsp;&nbsp;'''Vehicle Damage''' || '''120''' |- | '''''Chapter 5: Hardware''''' || '''''122''''' |- | &nbsp;&nbsp;'''Technology Key''' || '''123''' |- | &nbsp;&nbsp;'''Power Supplies''' || '''123''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Batteries'' || ''123'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fuel Cells'' || ''124'' |- | &nbsp;&nbsp;'''Electronics Components''' || '''125''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Access Chip'' || ''125'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Interactive Access Chip'' || ''125'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Interface Technology'' || ''125'' |- | &nbsp;&nbsp;'''Computers''' || '''126''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Body Computers'' || ''126'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dataspike'' || ''127'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Main Computers'' || ''127'' |- | &nbsp;&nbsp;'''Communications''' || '''127''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Personal Communications'' || ''127'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Uplink Communicators'' || ''127'' |- | &nbsp;&nbsp;'''Robotics and Remotes''' || '''128''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Robots'' || ''128'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Agricultural Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Construction Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Delivery Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Domestic Robot || 128 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Remotes'' || ''129'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cetacean Remote || 129 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Combat Remote || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Recon Remote || 130 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Surveillance Remote || 130 |- | &nbsp;&nbsp;'''Sensors''' || '''130''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Broad Spectrum Visor'' || ''130'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat HUD'' || ''130'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Digital Binoculars'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hand-Held Sonar Transceiver'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Security Monitors'' || ''131'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensor Array'' || ''131'' |- | &nbsp;&nbsp;'''Medical Gear''' || '''132''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Biomonitor'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Coagulant Pads'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cold-Sleep Capusules'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cerebral Cryo-Oxygenators'' || ''132'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Field Surgery Kit'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Standard Medical Kit'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ultrasonic Hypodermic'' || ''133'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Wound Glue'' || ''134'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Drugs'' || ''134'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Antibiotics/Antivirals || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anti-Shock || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Auntie Susie || 134 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Healing Booster || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Pain Inhibitor || 135 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Reflex Serum || 135 |- | &nbsp;&nbsp;'''Life Support''' || '''136''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artificial Gill'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Bacterial Life Support EVA System'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Diving Mask'' || ''136'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Drysuit'' || ''137'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Gill Suit'' || ''137'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hard Suit'' || ''137'' |- | &nbsp;&nbsp;'''Survival Gear''' || '''138''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Filter Mask'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Fire Paste'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Locator Beacon'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Portable Navigation Suite'' || ''138'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Rescue Ball'' || ''139'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Guide'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Grenade'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Survival Plastic'' || ''140'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Water Purifiers'' || ''140'' |- | &nbsp;&nbsp;'''Personal Propulsion''' || '''141''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''MHD Harness'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''MHD Sled'' || ''141'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Manoeuvre Jets'' || ''141'' |- | &nbsp;&nbsp;'''Remote Vehicle Technology''' || '''141''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Remotely Piloted Submersibles'' || ''141'' |- | &nbsp;&nbsp;'''Cetacean Technologies''' || '''142''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Artificial Gill'' || ''142'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''CICADA'' || ''142'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Cetacean Weapon Harness'' || ''143'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sonic Trode'' || ''143'' |- | &nbsp;&nbsp;'''Materials''' || '''143''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Bioplastic'' || ''143'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flex Grade || 143 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Foamed Plastic || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industrial Grade || 144 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Standard Grade || 144 |- | &nbsp;&nbsp;'''Body Armour''' || '''144''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Armour'' || ''144'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Combat Helmet'' || ''144'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Phototropic BDU'' || ''145'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Personal Body Armour'' || ''145'' |- | &nbsp;&nbsp;'''Melee Weapons''' || '''145''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Diamond Knife'' || ''145'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Stun Gloves and Batons'' || ''145'' |- | &nbsp;&nbsp;'''Firearms''' || '''146''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Handguns'' || ''146'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Concealable || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Large Calibre || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Small Calibre || 146 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Stunguns'' || ''147'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Longarms'' || ''148'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hunting Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Shotgun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sniper Rifle || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spear Gun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Submachine Gun || 148 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Heavy Weapons'' || ''149'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Cannon || 149 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Autocannon || 149 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Grenade Launcher || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Heavy Machinegun || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Light Machinegun || 150 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Torpedo Cannon || 150 |- | &nbsp;&nbsp;'''Firearm Accessories''' || '''151''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Ammunition'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour-Piercing Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Explosive Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flechette Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Standard Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Suppression Rounds || 151 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Assault Harness'' || ''151'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Laser Designator'' || ''152'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Optics'' || ''152'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Electronic Scopes || 152 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Optical Scopes || 152 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Targeting Interlink'' || ''152'' |- | &nbsp;&nbsp;'''Grenades''' || '''153''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Destructive Grenades'' || ''153'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour-Piercing Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Concussion Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Fragmentation Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Incendiary Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Suppression Grenades'' || ''153'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Adhesive Grenades || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Flash-Bang || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nausea Grenade || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sleep Grenade || 153 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Smoke Grenade || 153 |- | &nbsp;&nbsp;'''Vehicles''' || '''154''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Motorcycle'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hovercraft'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;APC Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Heavy Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Light Hovercraft || 155 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Jumpcraft'' || ''155'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Assault Jumpcraft || 156 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cargo Jumpcraft || 156 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Jumpbike || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Patrol Jumpcraft || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility Jumpcraft || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;''VTOL Aircraft'' || ''157'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility VTOL || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VTOL Strike Fighter || 157 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Submersibles'' || ''158'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Cetacean Power Shell || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Submersible Interceptor || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Research Submersible || 159 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Utility Submersible || 160 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Watercraft'' || ''160'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hydrofoil || 160 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Power Ski || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sailboats || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Vehicle Accessories'' || ''161'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armour || 161 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Auxiliary Fuel Tank || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Computer-Assisted Targeting System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ECM Suite || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Ejection System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Evac Pod || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Gunner's Turret || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Onboard Computer || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Radar Suite || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Security System || 162 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sonar Suite || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Targeting Computer || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;VR Cockpit || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Guided Weapons'' || ''163'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infantry Missile Launcher || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Multipurpose Anti-Vehicle Missile || 163 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Torpedo || 163 |- | '''''Chapter 6: Biotech''''' || '''''164''''' |- | &nbsp;&nbsp;'''Biotechnology''' || '''165''' |- | &nbsp;&nbsp;'''Cybernetic Modification''' || '''165''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Implant Computers'' || ''166'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Calculator || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Microcomputer || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Sensory Recorder || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Translator || 166 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Neural Jack'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Pain Inhibitor'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Programmed Reflexes'' || ''167'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Programs || 168 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensor and Technical Cyberwear'' || ''168'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Infrared Sensors || 168 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Radar || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Implanted Micro-Toolkit || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spyware || 169 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Anatomical Cyberwear'' || ''169'' |- | &nbsp;&nbsp;'''Genetic Redesign''' || '''170''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Aquaforms'' || ''170'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Hybrids'' || ''171'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Spacers'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Transhumans'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Genies and Biomods'' || ''173'' |- | &nbsp;&nbsp;'''Genetic Modification''' || '''173''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Medical Biomods'' || ''173'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Longevity Therapy || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Regeneration || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Metabolic Biomods'' || ''174'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anti-Poison || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Improved Blood Oxygenation || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Mind-Job || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Respiratory Filter || 174 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Multiglands || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Salt Tolerance || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Implanted Biomods'' || ''175'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Electro-Muscular Analogues || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Immunological Symbiote || 175 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Sensory Biomods'' || ''176'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Amplified Hearing || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Echolocation || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Enhanced Smell || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Extra Sensory Perception Enhancement || 176 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Telescopic Vision || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Full-Body Biomods'' || ''177'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Accelerated Neurons || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Aquaform || 177 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Body Sculpting || 178 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GEO Shock Trooper || 178 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hybrid || 179 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Spacer || 179 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Transhuman || 179 |- | '''''Chapter 7: Future History''''' | '''''180''''' |- | &nbsp;&nbsp;'''The Athena Project''' || '''181''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Prometheus II Returns || 184 |- | &nbsp;&nbsp;'''The Blight''' || '''184''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Europe'' || ''185'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Americas'' || ''187'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Africa and the Middle East'' || ''188'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Asia'' || ''189'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Antartica'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Earth Orbit'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Luna, Mars Colony, and the Asteroid Belt'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Death Toll'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Global Ecology Organization'' || ''190'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Shooting the Hole || 192 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;GEO Protectorates and Independents || 192 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Incorporate States'' || ''193'' |- | &nbsp;&nbsp;'''The Natives and the Abandonment''' || '''196''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: The Cooler || 198 |- | &nbsp;&nbsp;'''Recontact''' || '''199''' |- | &nbsp;&nbsp;'''The Long John Rush''' || '''200''' |- | &nbsp;&nbsp;'''On the Horizon''' || '''201''' |- | '''''Chapter 8: On the Frontier''''' || '''''202''''' |- | &nbsp;&nbsp;'''A World in Conflict''' || '''203''' |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Natives'' || ''203'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture and Psychology || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Native View || 203 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Relations between Diver and Squid || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Natives and Poseidon || 205 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;The Natives and Earth || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Technology || 206 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Native Settlements || 208 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Aquaculture and Agriculture || 209 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Lifestyle || 210 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Ritual and Ceremony || 211 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The GEO on Poseidon'' || ''212'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Human Resources || 212 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Natural Resources || 212 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Science and Technology || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Communications || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Internal Security || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;State and Internal Affairs || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Armed Forces || 213 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Justice || 214 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Trade and Industry || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Incorporate States on Poseidon'' || ''216'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anasi Systems || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Atlas Materials || 216 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Biogene || 217 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Long John || 217 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dundalk Shipbuilding || 218 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Gendiver || 218 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hanover Industries || 220 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Hydrospan || 221 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Lavender Organics || 222 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MacLeod Enforcement || 223 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Nippon Industrial State || 224 |- | &nbsp;&nbsp;&nbsp;&nbsp;''The Newcomers'' || ''224'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Pioneers || 225 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Box: Colonial Calendar || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Prospectors || 226 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Opportunists || 227 |- | &nbsp;&nbsp;'''Cetaceans''' || '''227''' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;History || 227 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Life on Poseidon || 228 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Dolphins'' || ''229'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anatomy and Physiology || 229 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Psychology || 229 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sociology || 230 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture || 231 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Killer Whales'' || ''232'' |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Anatomy and Physiology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Psychology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sociology || 232 |- | &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Culture || 233 |- | &nbsp;&nbsp;&nbsp;&nbsp;''Technology'' || ''233'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Lifestyle'' || ''234'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Transportation'' || ''234'' |- | &nbsp;&nbsp;&nbsp;&nbsp;''Communication'' || ''234'' |- | '''''Chapter 9: Timeline''''' || '''''236''''' |} [[Category:Blue Planet]] 822649943ea8fb6ace3ecaf8f05ee0aa36190695 0 1540 1794 1793 2008-07-21T11:28:12Z Neil 2 Added MySQL backup user wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <tt>/etc/cron.daily/makedailybackups</tt>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +1 -exec rm -f {} \; and <tt>/etc/cron.monthly/makemonthlybackups</tt> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -ctime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <tt>--absolute-names</tt> would restore the files to paths anchored at root. == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <tt>/etc/cron.daily/makedailybackups</tt>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +2 -exec rm -f {} \; # Create file backups tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null [[Category:Server setup]] cf04534894623fece9646c6a336f5f9063d582c3 1795 1794 2008-07-21T11:31:15Z Neil 2 /* Create users */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <tt>/etc/cron.daily/makedailybackups</tt>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +1 -exec rm -f {} \; and <tt>/etc/cron.monthly/makemonthlybackups</tt> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -ctime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <tt>--absolute-names</tt> would restore the files to paths anchored at root. == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <tt>/etc/cron.daily/makedailybackups</tt>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +2 -exec rm -f {} \; # Create file backups tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null [[Category:Server setup]] 580cc11b40eb9146f244064c49e74dbd6192690b 0 1541 1796 1785 2008-07-21T12:43:10Z Neil 2 /* Regenerating OpenSSH keys */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <tt>/etc/hostname</tt> to server.domain.tld == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 * Change <tt>/etc/resolv.conf</tt> to nameserver 192.168.1.1 (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; [[Category:Server setup]] 7aeea6c8a11b613fa396feaf32a8a1705ab54e78 0 1538 1797 1768 2008-07-21T14:44:23Z Neil 2 wikitext text/x-wiki * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> wget http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz tar -xvzf rubygems-1.1.1.tgz ~$ rm rubygems-1.1.1.tgz ~$ cd rubygems-1.1.1 ~$ sudo ruby setup.rb ~$ cd .. ~$ rm -r rubygems-1.1.1 (DO NOT use sudo ruby rubygemsx.x.x/setup.rb) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster (Don't need <tt>mongrel_cluster</tt> on the desktop machine.) d5af91c3a8775e251dbab64a9934199767b4e6b3 1798 1797 2008-07-21T14:44:53Z Neil 2 wikitext text/x-wiki * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> tar -xvzf rubygems-1.1.1.tgz ~$ rm rubygems-1.1.1.tgz ~$ cd rubygems-1.1.1 ~$ sudo ruby setup.rb ~$ cd .. ~$ rm -r rubygems-1.1.1 (DO NOT use sudo ruby rubygemsx.x.x/setup.rb) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster (Don't need <tt>mongrel_cluster</tt> on the desktop machine.) 27c0e44247296d78ab1ca6d546a7ac4eca1fc156 1799 1798 2008-07-21T15:22:18Z Neil 2 wikitext text/x-wiki * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# sudo ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster (Don't need <tt>mongrel_cluster</tt> on the desktop machine.) 93929298ad8a58f3a03dc00757e19a933fdca102 1801 1799 2008-07-22T18:14:40Z Neil 2 wikitext text/x-wiki * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# sudo ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. 483bb09a5cd625c42bcd5516d56badd1ef31a076 1804 1801 2008-07-23T10:02:26Z Neil 2 wikitext text/x-wiki * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. 7fc068d7cd245c589df8a97cbe8836040fc984c9 1805 1804 2008-07-23T10:23:11Z Neil 2 wikitext text/x-wiki * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. 5a93f91ca85b77ad123c5339decd842c38860edd 1807 1805 2008-07-23T13:21:58Z Neil 2 wikitext text/x-wiki == Installation of Rails == * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 6e33ed4e4abf19336b49fc712234c457c7960929 0 1536 1800 1766 2008-07-21T19:07:02Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Backup setup|Backup]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <tt>/etc/samba/smb.conf</tt> (note that you'll have to skip around the file to find all the settings) ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. [[Category:Server setup]] b0e3f5d290c135bbe38ba7224a57c3e8c0355827 1834 1800 2008-10-09T19:35:24Z Neil 2 Added password sync = no wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Backup setup|Backup]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <tt>/etc/samba/smb.conf</tt> (note that you'll have to skip around the file to find all the settings) [global] ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of workgroup = myworkgroup ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. unix password sync = no #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. [[Category:Server setup]] 84939cf246548a797b854a78c768d0230881dce6 0 1526 1802 1749 2008-07-22T22:30:22Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='la -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == build-essential kicker-applets [[Category:Server setup]] f8b48049a139f62f6b905261427cc5950536bbee 1803 1802 2008-07-23T09:08:34Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='la -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == build-essential kicker-applets netbeans [[Category:Server setup]] f3e6d81fd94239f204806c01d53d5472eadb9185 1806 1803 2008-07-23T11:20:33Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='la -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == build-essential kicker-applets netbeans == Password-less SSH logins == Follow [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html these instructions]. [[Category:Server setup]] 1cb9aab1a67c1e5bb500b2dcfd802d21a8113002 1808 1806 2008-07-23T21:28:43Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='la -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == build-essential kicker-applets netbeans == Password-less SSH logins == Follow [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html these instructions]. == Installing and using Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 6274ddb84dc6cb561abbf8425f3e6e77713b996d 1810 1808 2008-07-28T09:53:19Z Neil 2 /* Additional packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='la -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == apt-get install build-essential kicker-applets Install from source: Netbeans VMWare client Google Earth == Password-less SSH logins == Follow [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html these instructions]. == Installing and using Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 1c624d00fa53d950d7fc24eafeaeea690544d334 1832 1810 2008-09-24T08:54:31Z Neil 2 /* Additional packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='la -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == root@desktop:~# apt-get install build-essential kicker-applets Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. Install from source: * Netbeans * VMWare client * Google Earth == Password-less SSH logins == Follow [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html these instructions]. == Installing and using Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 270b7aaf54bee5907bd585eb4f4add173672fd2a 1833 1832 2008-09-24T08:55:36Z Neil 2 /* Command aliases */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == root@desktop:~# apt-get install build-essential kicker-applets Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. Install from source: * Netbeans * VMWare client * Google Earth == Password-less SSH logins == Follow [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html these instructions]. == Installing and using Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] f7780caec0cbe87eb6b150f73d07dd12ccae385e 1835 1833 2008-10-16T10:04:32Z Neil 2 /* Password-less SSH logins */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == root@desktop:~# apt-get install build-essential kicker-applets Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. Install from source: * Netbeans * VMWare client * Google Earth == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Installing and using Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 094129a4a3be819c94a0da55ad992b8113203bf2 1837 1835 2008-11-17T23:07:38Z Neil 2 How to make Konqueror the default file manager wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == root@desktop:~# apt-get install build-essential kicker-applets Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. Install from source: * Netbeans * VMWare client * Google Earth == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Making Konqueror default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror<code> up the list to make it the preferred application for opening such things. == Installing and using Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 55be51f845573c8cb361f7127208eed60f9076f8 1838 1837 2008-11-17T23:08:06Z Neil 2 Fixed typo wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == root@desktop:~# apt-get install build-essential kicker-applets Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. Install from source: * Netbeans * VMWare client * Google Earth == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Making Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror<code> up the list to make it the preferred application for opening such things. == Installing and using Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 5eff9603c3d99474ac88e7e90e05f79e2541e87e 1839 1838 2008-11-17T23:10:07Z Neil 2 Another typo fix wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Additional packages == root@desktop:~# apt-get install build-essential kicker-applets Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. Install from source: * Netbeans * VMWare client * Google Earth == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 60afce83789ede3abaacf3391a8574616ff400fa 1841 1839 2008-11-19T10:10:56Z Neil 2 Added installed package list wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. First, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * acrobat reader **** * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 * ghc6-prof * haskell-mode * python * perl * ruby Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] b0d75c5e3e9ff060bd72668652c061897d7fc157 1842 1841 2008-11-19T10:32:05Z Neil 2 /* Install additional packages */ Added Firefox extensions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * acrobat reader **** * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 * ghc6-prof * haskell-mode * python * perl * ruby === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 52d0592cff7466fe2297f7475232fcfbcb33e151 1843 1842 2008-11-19T11:03:25Z Neil 2 /* Install additional packages */ Added a few more wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * acrobat reader **** * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * smu-cl * slime * kile * jabref * kbibtex === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] f176f32357f04b639eb875b88ab80c0a57f40e6a 1844 1843 2008-11-19T12:13:54Z Neil 2 /* Install additional packages */ Added Medibuntu notes wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * smu-cl * slime * kile * jabref * kbibtex === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 15f158f0dce13598df93ff6ad5e82504292ca398 1845 1844 2008-11-19T12:31:55Z Neil 2 /* Normal packages */ Switched Lisp implementation wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 4778dbf1ac5156940e1d51a3469ccfd805ad3d31 0 1543 1809 1782 2008-07-23T21:33:28Z Neil 2 How to remove ufw wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <tt>/proc</tt> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to remove the default <tt>ufw</tt> package. However, removing it with <tt>apt-get</tt> breaks the <tt>ubuntu-standard</tt> package, which is probably not a good idea. Instead, simply remove the startup scripts from the <tt>/etc/rc.*</tt> directories. * Remove the main script: root@server:~# rm /etc/init.d/ufw * Remove all the auto-start and stop scripts: root@server:~# update-rc.d ufw remove With that done, you can move on to installing a more sensible firewall. == Add a bespoke iptables firewall == * Add the <tt>/etc/iptables.rules</tt> file and the <tt>/etc/iptables</tt> file. Adjust <tt>/etc/iptables.rules</tt> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> === Iptables.rules === This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END === Iptables script === #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] c82b8d331cc1ee712513e425d8121b6c7fad18da 1840 1809 2008-11-17T23:16:51Z Neil 2 Changed to a more sensible method of disabling UFW wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> === Iptables.rules === This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END === Iptables script === #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 5cc34a80038f9628e34a5ffe2cb7c2723a941d6e 0 1595 1811 1604 2008-08-21T19:34:59Z Neil 2 Updated fees wikitext text/x-wiki {| align="right" {{Prettytable}} ! {{Hl3}}|'''At A Glance''' |- | * Chemistry tutoring * A-level specialist * Based in Milton Keynes * £30.00 per hour * Email: [mailto:tutor@njae.me.uk tutor@njae.me.uk] |} I, [mailto:tutor@njae.me.uk Jenni Smith], am available to give private tuition in Chemistry at A-level (AS and A2). This tuition can give great support for students, especially for more gifted students who require an additional challenge to achieve their best, or those who could take advantage of extra guidance to achieve higher examination grades. I normally tutor in addition to their normal studies at college or school. My style of tuition is student-led; I concentrate on the areas where the student wants assistance. Most students do well with one or two hours tuition per week, plus some additional work that I set. The earlier tuition starts, the more benefit will be gained. I tutor in the evenings at my house in Milton Keynes. Tutees have come from the area, and from as far afield as Northampton, Wellingborough, and Leighton Buzzard. Please [mailto:tutor@njae.me.uk contact me] if you would like more information, or to arrange tuition. == Subjects and Levels Taught == I tutor Chemistry at A-level. I can give tuition for both AS-level and A2-level. I have experience with all exam board syllabi, including Nuffield, Edexcel, AQA, and OCR. == Fees == My fees are £30.00 per hour. == Experience == I am a DfEE qualified teacher with several years' experience of teaching in secondary schools; a large portion of my teaching time was spent teaching Chemistry A-level. I am now teaching special needs children. I have been tutoring AS and A2 students for over six years and still enjoy it. I am also an Assistant Examiner for Chemistry A-level, working for the Edexcel board. As with all people who work with children, I have a valid Criminal Records Bureau certificate. References are available on request. 6d17189484d3ca243ae24c44997c2388514c3114 0 1 1812 1684 2008-09-04T20:56:03Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name Generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference). == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 91b51d0f4c05ceeb4b1db572aed73db176877b5d 1822 1812 2008-09-06T21:38:03Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference). == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 2d39d95d884bfd3e8d7a531e3644d46dfaa0da53 1823 1822 2008-09-06T21:41:28Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference). I also have a few [[downloadable programs]] (nothing exciting). == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 9999a25a3a9a0ad04226433bfc2c16080a4a1b74 0 1549 1814 2008-09-04T21:25:13Z Neil 2 New page: ''(This was written by Michael Harvey to support his [[:Category:Name Generation|name generation]] program.)'' Made-up names, especially those found in literature and roleplaying games, o... wikitext text/x-wiki ''(This was written by Michael Harvey to support his [[:Category:Name Generation|name generation]] program.)'' Made-up names, especially those found in literature and roleplaying games, often sound silly, out of place, or not-quite right. To be honest, it can be difficult thinking up creative names that sound right. Names is a program which can generate hundreds of names based on the input you feed it. Because it combines "elements" from data files to form names, it is possible to generate very high quality output. Further, it is possible to create data files which provide a "feel" for a particular language or culture -- real or imaginary. This enables the writer or game designer to genarate lists of names for each fantasy culture, which all have a certain sound and seem to belong together. This document has various suggestions on how to create element files, as well as how to generate good quality names. == Names elements == There are several types of name elements: ;Prefix (PRE) :Parts that begin a word. Example: "c", "co", and "con" could all be prefixes for "Conan" ;Middle (MID) :Parts that go in the middle of a word, BETWEEN a prefix and suffix. Example: "cam" is a middle for "Alcamtar", where "al" is the prefix and "tar" is the suffix ;Suffix (SUF) :Parts that end a word. In the name Aragorn, suffixes could be "agorn", "gorn", "orn", or even "n" ;Noun (NOUN) :Nouns: "road", "fortress", "tree", "crossing" ;Adjective (ADJ) :Adjectives: "gray", "slow", "stubborn" ;Noun/adjective (NADJ) :These are words which could be either nouns or adjectives. Examples: dark ("darkwood", "outer dark"), silver ("silverlode", "moria-silver"), wood ("woodtown", "goblin-wood") :Many nouns and adjectives actually fall into this category. This gives the program more flexibility in using them, providing much more variety in names. Elements should not be duplicated in two categories. For example, "orc" should not be under both noun and adjective, or you might get something like "Orcorc". Instead, make it a NADJ and it won't get used twice. Elements that you want to emphasize may be duplicated within the same category. For instance, if you list "forest" three times in a row, it is three times more likely to be selected. Elements are stored in plain text .ELE files. Thus, you might have one file of goblin sounding names and another for elvish words, etc. == Element files == These should have the extension <code>.ELE</code>. The element file contains several types of elements, denoted by simple abbreviations: {| |<code>PRE</code>||prefixes |- |<code>MID</code>||middles |- |<code>SUF</code>||suffixes |- |<code>NOUN</code>||nouns |- |<code>ADJ</code>||adjectives |- |<code>NADJ</code>||noun/adjectives |- |<code>END</code>||this signifies the end of the file |} The pound sign (#) is used as an escape character to denote the start of a new section (much like preprocessor directives in C). Thus, a typical file would look like this: #PRE con Conan gal Galan tar Example Taran #SUF Output: Galarok an Conarok arok Tararok #END This defines three prefixes (con,gal,tar) and two suffixes (an,arok). Notice the #END used to end the file. There should not be any blank lines in the file, unless you want to define empty parts. Parts may include any character, including spaces. For example, #PRE con gal tar #SUF an 'Arok -Rog e arba zud #END Might produce the following output: Con An Gal'Arok Galzud Tar-Rog Cone Arba Elements in an element file should not be capitalized. The program will automatically capitalize the names after each space. If you do put any capitals in, they will not be converted to lowercase, so this could be used to artificially capitalize words (usually after special punctuation, as above). The exception to this general "rule" is when you want to capitalize an element after a punctuation mark; in the example above, 'Arok would not have been automatically capitalized because there is no whitespace. I find it helpful to keep all the elements in each section alphabetized so that I can check for duplicates. USE OF ELEMENT FILES The prefix/suffix form is generally the most useful. Most of the example files are in this form. Note that you are not restricted to fantasy names. By using english words you can get things like "Darkwood", "Brightblade", "Millennium Falcon", "Nottingham" and so forth. See the sample files for ideas. Middles can add a lot of variety to your names, but they also make the names themselves longer. Noun/NADJ/ADJ elements are mostly useful for idea-generators, or special places like "House of the Skulls" or "The Books of the Priest." The program combines these in a variety of ways, and also pluralizes them sometimes. All you need to do is supply the list of words. Such lists may represent themes, such as dark fantasy, science fiction, nautical things, wizardly things, and so forth. Again, see the sample ELE files. It is easy to make your own ELE files. There are several methods I have found personally useful. One is to just sit down and starting making up the elements out of your head. Just come up with all the combinations you can think of and put them in. Another is to compile a list of names from your favorite book(s), movie(s), etc. You can then easily split these names into prefix/suffix parts. Example: PRE SUF Alcamtar alcam tar Celowin cel owin Conan ==> con an Gimli gim li Talward tal ward One helpful strategy in making your names sound better is to end the prefixes and begin the suffixes consistently. Either end the prefixes with a vowel and begin the suffixes with a consonant, or end the prefixes with a consonant and begin the suffixes with a vowel. I generally prefer the latter. Example: METHOD #1 METHOD #2 Alcamtar alca / mtar alcam / tar Celowin celo / win cel / owin Conan co / nan con / an Gimli gi / mli gim / li Talward talwa / rd tal / ward This technique will keep your names from sounding too random, and will make them more pronounceable. Note that soft sounds such as r, l, t, s, w, y might be including with the vowels, as I have done above. Use your imagination and common sense, and put a few together yourself to see how they sound. Another helpful technique is to make all the words of a given language use the same set of sounds, or same spelling. This might require a little research. Some good references are the appendices to the Lord of the Rings, or an introductory book on phonetics/language from the library. Some simple examples: Elvish spelling: 'C' 'G' 'DH' 'T' 'S' 'W' 'F' Orcish spelling: 'K' 'GH' 'J' 'D' 'Z' 'V' 'V' Here is a chart. I'm not going to take the time to try an explain it, but study it and compare the sounds to one another. You will notice patterns. It probably isn't too accurate, but is still useful. (yes, I got the idea from Tolkein.) ---STOPS--- -SPIRANTS-- --OTHER---- ---NASAL--- Soft Voiced Soft Voiced Soft Voiced Soft Voiced Labial p b bh wh w mh m Labiodental f v y Dental th th Palatal? t d ch j lh l nh n Sibilant s z sh zh rh r Alveolar? c,q,k g ch gh h h ng You can use this to create "sounds" for languages. For instance, celtic has a lot of unvoiced (soft) sounds, making it sound smooth and soft and flowing. German, on the other hand, has a lot of voiced stops, making it louder and harsher sounding. By selecting which sounds are most common, you can give each fantasy language a certain "feel." One handy technique is to get an atlas of foreign countries and look at the names on the map. You can get a pretty good idea of which sounds are common, and how they should go together this way. This is the way I did my "french" ELE file, and also the "aztec" and "african" ones. CONCLUSION Hope you find this useful. I can be contacted for comments/questions on the Internet as "mike@cs.pdx.edu". 02aee89fb3adbbd8c96e0091033a0f59d026bf58 1815 1814 2008-09-04T21:27:20Z Neil 2 wikitext text/x-wiki ''(This was written by Michael Harvey to support his [[:Category:Name Generation|name generation]] program.)'' Made-up names, especially those found in literature and roleplaying games, often sound silly, out of place, or not-quite right. To be honest, it can be difficult thinking up creative names that sound right. Names is a program which can generate hundreds of names based on the input you feed it. Because it combines "elements" from data files to form names, it is possible to generate very high quality output. Further, it is possible to create data files which provide a "feel" for a particular language or culture -- real or imaginary. This enables the writer or game designer to genarate lists of names for each fantasy culture, which all have a certain sound and seem to belong together. This document has various suggestions on how to create element files, as well as how to generate good quality names. == Names elements == There are several types of name elements: ;Prefix (PRE) :Parts that begin a word. Example: "c", "co", and "con" could all be prefixes for "Conan" ;Middle (MID) :Parts that go in the middle of a word, ''between'' a prefix and suffix. Example: "cam" is a middle for "Alcamtar", where "al" is the prefix and "tar" is the suffix ;Suffix (SUF) :Parts that end a word. In the name Aragorn, suffixes could be "agorn", "gorn", "orn", or even "n" ;Noun (NOUN) :Nouns: "road", "fortress", "tree", "crossing" ;Adjective (ADJ) :Adjectives: "gray", "slow", "stubborn" ;Noun/adjective (NADJ) :These are words which could be either nouns or adjectives. Examples: dark ("darkwood", "outer dark"), silver ("silverlode", "moria-silver"), wood ("woodtown", "goblin-wood") :Many nouns and adjectives actually fall into this category. This gives the program more flexibility in using them, providing much more variety in names. Elements should not be duplicated in two categories. For example, "orc" should not be under both noun and adjective, or you might get something like "Orcorc". Instead, make it a NADJ and it won't get used twice. Elements that you want to emphasize may be duplicated within the same category. For instance, if you list "forest" three times in a row, it is three times more likely to be selected. Elements are stored in plain text .ELE files. Thus, you might have one file of goblin sounding names and another for elvish words, etc. == Element files == These should have the extension <code>.ELE</code>. The element file contains several types of elements, denoted by simple abbreviations: {| |<code>PRE</code>||prefixes |- |<code>MID</code>||middles |- |<code>SUF</code>||suffixes |- |<code>NOUN</code>||nouns |- |<code>ADJ</code>||adjectives |- |<code>NADJ</code>||noun/adjectives |- |<code>END</code>||this signifies the end of the file |} The pound sign (#) is used as an escape character to denote the start of a new section (much like preprocessor directives in C). Thus, a typical file would look like this: #PRE con Conan gal Galan tar Example Taran #SUF Output: Galarok an Conarok arok Tararok #END This defines three prefixes (con,gal,tar) and two suffixes (an,arok). Notice the #END used to end the file. There should not be any blank lines in the file, unless you want to define empty parts. Parts may include any character, including spaces. For example, #PRE con gal tar #SUF an 'Arok -Rog e arba zud #END Might produce the following output: Con An Gal'Arok Galzud Tar-Rog Cone Arba Elements in an element file should not be capitalized. The program will automatically capitalize the names after each space. If you do put any capitals in, they will not be converted to lowercase, so this could be used to artificially capitalize words (usually after special punctuation, as above). The exception to this general "rule" is when you want to capitalize an element after a punctuation mark; in the example above, 'Arok would not have been automatically capitalized because there is no whitespace. I find it helpful to keep all the elements in each section alphabetized so that I can check for duplicates. USE OF ELEMENT FILES The prefix/suffix form is generally the most useful. Most of the example files are in this form. Note that you are not restricted to fantasy names. By using english words you can get things like "Darkwood", "Brightblade", "Millennium Falcon", "Nottingham" and so forth. See the sample files for ideas. Middles can add a lot of variety to your names, but they also make the names themselves longer. Noun/NADJ/ADJ elements are mostly useful for idea-generators, or special places like "House of the Skulls" or "The Books of the Priest." The program combines these in a variety of ways, and also pluralizes them sometimes. All you need to do is supply the list of words. Such lists may represent themes, such as dark fantasy, science fiction, nautical things, wizardly things, and so forth. Again, see the sample ELE files. It is easy to make your own ELE files. There are several methods I have found personally useful. One is to just sit down and starting making up the elements out of your head. Just come up with all the combinations you can think of and put them in. Another is to compile a list of names from your favorite book(s), movie(s), etc. You can then easily split these names into prefix/suffix parts. Example: PRE SUF Alcamtar alcam tar Celowin cel owin Conan ==> con an Gimli gim li Talward tal ward One helpful strategy in making your names sound better is to end the prefixes and begin the suffixes consistently. Either end the prefixes with a vowel and begin the suffixes with a consonant, or end the prefixes with a consonant and begin the suffixes with a vowel. I generally prefer the latter. Example: METHOD #1 METHOD #2 Alcamtar alca / mtar alcam / tar Celowin celo / win cel / owin Conan co / nan con / an Gimli gi / mli gim / li Talward talwa / rd tal / ward This technique will keep your names from sounding too random, and will make them more pronounceable. Note that soft sounds such as r, l, t, s, w, y might be including with the vowels, as I have done above. Use your imagination and common sense, and put a few together yourself to see how they sound. Another helpful technique is to make all the words of a given language use the same set of sounds, or same spelling. This might require a little research. Some good references are the appendices to the Lord of the Rings, or an introductory book on phonetics/language from the library. Some simple examples: Elvish spelling: 'C' 'G' 'DH' 'T' 'S' 'W' 'F' Orcish spelling: 'K' 'GH' 'J' 'D' 'Z' 'V' 'V' Here is a chart. I'm not going to take the time to try an explain it, but study it and compare the sounds to one another. You will notice patterns. It probably isn't too accurate, but is still useful. (yes, I got the idea from Tolkein.) ---STOPS--- -SPIRANTS-- --OTHER---- ---NASAL--- Soft Voiced Soft Voiced Soft Voiced Soft Voiced Labial p b bh wh w mh m Labiodental f v y Dental th th Palatal? t d ch j lh l nh n Sibilant s z sh zh rh r Alveolar? c,q,k g ch gh h h ng You can use this to create "sounds" for languages. For instance, celtic has a lot of unvoiced (soft) sounds, making it sound smooth and soft and flowing. German, on the other hand, has a lot of voiced stops, making it louder and harsher sounding. By selecting which sounds are most common, you can give each fantasy language a certain "feel." One handy technique is to get an atlas of foreign countries and look at the names on the map. You can get a pretty good idea of which sounds are common, and how they should go together this way. This is the way I did my "french" ELE file, and also the "aztec" and "african" ones. CONCLUSION Hope you find this useful. I can be contacted for comments/questions on the Internet as "mike@cs.pdx.edu". a10d0d03841ee858310bcbf7451a0b03e009e98b 1816 1815 2008-09-06T14:05:09Z Neil 2 wikitext text/x-wiki ''(This was written by Michael Harvey to support his [[:Category:Name Generation|name generation]] program.)'' Made-up names, especially those found in literature and roleplaying games, often sound silly, out of place, or not-quite right. To be honest, it can be difficult thinking up creative names that sound right. Names is a program which can generate hundreds of names based on the input you feed it. Because it combines "elements" from data files to form names, it is possible to generate very high quality output. Further, it is possible to create data files which provide a "feel" for a particular language or culture -- real or imaginary. This enables the writer or game designer to generate lists of names for each fantasy culture, which all have a certain sound and seem to belong together. This document has various suggestions on how to create element files, as well as how to generate good quality names. == Names elements == There are several types of name elements: ;Prefix (PRE) :Parts that begin a word. Example: "c", "co", and "con" could all be prefixes for "Conan" ;Middle (MID) :Parts that go in the middle of a word, ''between'' a prefix and suffix. Example: "cam" is a middle for "Alcamtar", where "al" is the prefix and "tar" is the suffix ;Suffix (SUF) :Parts that end a word. In the name Aragorn, suffixes could be "agorn", "gorn", "orn", or even "n" ;Noun (NOUN) :Nouns: "road", "fortress", "tree", "crossing" ;Adjective (ADJ) :Adjectives: "gray", "slow", "stubborn" ;Noun/adjective (NADJ) :These are words which could be either nouns or adjectives. Examples: dark ("darkwood", "outer dark"), silver ("silverlode", "moria-silver"), wood ("woodtown", "goblin-wood") :Many nouns and adjectives actually fall into this category. This gives the program more flexibility in using them, providing much more variety in names. Elements should not be duplicated in two categories. For example, "orc" should not be under both noun and adjective, or you might get something like "Orcorc". Instead, make it a NADJ and it won't get used twice. Elements that you want to emphasize may be duplicated within the same category. For instance, if you list "forest" three times in a row, it is three times more likely to be selected. Elements are stored in plain text .ELE files. Thus, you might have one file of goblin sounding names and another for elvish words, etc. == Element files == These should have the extension <code>.ELE</code>. The element file contains several types of elements, denoted by simple abbreviations: {| |<code>PRE</code>||prefixes |- |<code>MID</code>||middles |- |<code>SUF</code>||suffixes |- |<code>NOUN</code>||nouns |- |<code>ADJ</code>||adjectives |- |<code>NADJ</code>||noun/adjectives |- |<code>END</code>||this signifies the end of the file |} The pound sign (#) is used as an escape character to denote the start of a new section (much like preprocessor directives in C). Thus, a typical file would look like this: #PRE con gal tar #SUF an arok #END Example output: Conan Galan Taran Galarok Conarok Tararok This defines three prefixes (con,gal,tar) and two suffixes (an,arok). Notice the #END used to end the file. There should not be any blank lines in the file, unless you want to define empty parts. Parts may include any character, including spaces. For example, #PRE con gal tar #SUF an 'Arok -Rog e arba zud #END Might produce the following output: Con An Gal'Arok Galzud Tar-Rog Cone Arba Elements in an element file should not be capitalized. The program will automatically capitalize the names after each space. If you do put any capitals in, they will not be converted to lowercase, so this could be used to artificially capitalize words (usually after special punctuation, as above). The exception to this general "rule" is when you want to capitalize an element after a punctuation mark; in the example above, 'Arok would not have been automatically capitalized because there is no whitespace. I find it helpful to keep all the elements in each section alphabetized so that I can check for duplicates. == Use of element files == The prefix/suffix form is generally the most useful. Most of the example files are in this form. Note that you are not restricted to fantasy names. By using english words you can get things like "Darkwood", "Brightblade", "Millennium Falcon", "Nottingham" and so forth. See the sample files for ideas. Middles can add a lot of variety to your names, but they also make the names themselves longer. Noun/NADJ/ADJ elements are mostly useful for idea-generators, or special places like "House of the Skulls" or "The Books of the Priest." The program combines these in a variety of ways, and also pluralizes them sometimes. All you need to do is supply the list of words. Such lists may represent themes, such as dark fantasy, science fiction, nautical things, wizardly things, and so forth. Again, see the sample ELE files. It is easy to make your own ELE files. There are several methods I have found personally useful. One is to just sit down and starting making up the elements out of your head. Just come up with all the combinations you can think of and put them in. Another is to compile a list of names from your favorite book(s), movie(s), etc. You can then easily split these names into prefix/suffix parts. Example: {| ! Original name !! PRE !! SUF |- | Alcamtar || alcam || tar |- | Celowin || cel || owin |- | Conan || con || an |- | Gimli || gim || li |- | Talward || tal || ward |} One helpful strategy in making your names sound better is to end the prefixes and begin the suffixes consistently. Either end the prefixes with a vowel and begin the suffixes with a consonant, or end the prefixes with a consonant and begin the suffixes with a vowel. I generally prefer the latter. Example: {| ! !! Method #1 !! Method #2 |- | Alcamtar || alca / mtar || alcam / tar |- | Celowin || celo / win || cel / owin |- | Conan || co / nan || con / an |- | Gimli || gi / mli || gim / li |- | Talward || talwa / rd || tal / ward |} This technique will keep your names from sounding too random, and will make them more pronounceable. Note that soft sounds such as r, l, t, s, w, y might be including with the vowels, as I have done above. Use your imagination and common sense, and put a few together yourself to see how they sound. Another helpful technique is to make all the words of a given language use the same set of sounds, or same spelling. This might require a little research. Some good references are the appendices to the Lord of the Rings, or an introductory book on phonetics/language from the library. Some simple examples: Elvish spelling: 'C' 'G' 'DH' 'T' 'S' 'W' 'F' Orcish spelling: 'K' 'GH' 'J' 'D' 'Z' 'V' 'V' Here is a chart. I'm not going to take the time to try an explain it, but study it and compare the sounds to one another. You will notice patterns. It probably isn't too accurate, but is still useful. (yes, I got the idea from Tolkein.) ---STOPS--- -SPIRANTS-- --OTHER---- ---NASAL--- Soft Voiced Soft Voiced Soft Voiced Soft Voiced Labial p b bh wh w mh m Labiodental f v y Dental th th Palatal? t d ch j lh l nh n Sibilant s z sh zh rh r Alveolar? c,q,k g ch gh h h ng You can use this to create "sounds" for languages. For instance, celtic has a lot of unvoiced (soft) sounds, making it sound smooth and soft and flowing. German, on the other hand, has a lot of voiced stops, making it louder and harsher sounding. By selecting which sounds are most common, you can give each fantasy language a certain "feel." One handy technique is to get an atlas of foreign countries and look at the names on the map. You can get a pretty good idea of which sounds are common, and how they should go together this way. This is the way I did my "french" ELE file, and also the "aztec" and "african" ones. CONCLUSION Hope you find this useful. I can be contacted for comments/questions on the Internet as "mike@cs.pdx.edu". a339861a4c24f9842a685464fa60801d3f7cea6b 1817 1816 2008-09-06T14:43:00Z Neil 2 wikitext text/x-wiki ''(This was written by Michael Harvey to support his [[:Category:Name Generation|name generation]] program.)'' Made-up names, especially those found in literature and roleplaying games, often sound silly, out of place, or not-quite right. To be honest, it can be difficult thinking up creative names that sound right. Names is a program which can generate hundreds of names based on the input you feed it. Because it combines "elements" from data files to form names, it is possible to generate very high quality output. Further, it is possible to create data files which provide a "feel" for a particular language or culture -- real or imaginary. This enables the writer or game designer to generate lists of names for each fantasy culture, which all have a certain sound and seem to belong together. This document has various suggestions on how to create element files, as well as how to generate good quality names. == Names elements == There are several types of name elements: ;Prefix (PRE) :Parts that begin a word. Example: "c", "co", and "con" could all be prefixes for "Conan" ;Middle (MID) :Parts that go in the middle of a word, ''between'' a prefix and suffix. Example: "cam" is a middle for "Alcamtar", where "al" is the prefix and "tar" is the suffix ;Suffix (SUF) :Parts that end a word. In the name Aragorn, suffixes could be "agorn", "gorn", "orn", or even "n" ;Noun (NOUN) :Nouns: "road", "fortress", "tree", "crossing" ;Adjective (ADJ) :Adjectives: "gray", "slow", "stubborn" ;Noun/adjective (NADJ) :These are words which could be either nouns or adjectives. Examples: dark ("darkwood", "outer dark"), silver ("silverlode", "moria-silver"), wood ("woodtown", "goblin-wood") :Many nouns and adjectives actually fall into this category. This gives the program more flexibility in using them, providing much more variety in names. Elements should not be duplicated in two categories. For example, "orc" should not be under both noun and adjective, or you might get something like "Orcorc". Instead, make it a NADJ and it won't get used twice. Elements that you want to emphasize may be duplicated within the same category. For instance, if you list "forest" three times in a row, it is three times more likely to be selected. Elements are stored in plain text .ELE files. Thus, you might have one file of goblin sounding names and another for elvish words, etc. == Element files == These should have the extension <code>.ELE</code>. The element file contains several types of elements, denoted by simple abbreviations: {| |<code>PRE</code>||prefixes |- |<code>MID</code>||middles |- |<code>SUF</code>||suffixes |- |<code>NOUN</code>||nouns |- |<code>ADJ</code>||adjectives |- |<code>NADJ</code>||noun/adjectives |- |<code>END</code>||this signifies the end of the file |} The pound sign (#) is used as an escape character to denote the start of a new section (much like preprocessor directives in C). Thus, a typical file would look like this: #PRE con gal tar #SUF an arok #END Example output: Conan Galan Taran Galarok Conarok Tararok This defines three prefixes (con,gal,tar) and two suffixes (an,arok). Notice the #END used to end the file. There should not be any blank lines in the file, unless you want to define empty parts. Parts may include any character, including spaces. For example, #PRE con gal tar #SUF an 'Arok -Rog e arba zud #END Might produce the following output: Con An Gal'Arok Galzud Tar-Rog Cone Arba Elements in an element file should not be capitalized. The program will automatically capitalize the names after each space. If you do put any capitals in, they will not be converted to lowercase, so this could be used to artificially capitalize words (usually after special punctuation, as above). The exception to this general "rule" is when you want to capitalize an element after a punctuation mark; in the example above, 'Arok would not have been automatically capitalized because there is no whitespace. I find it helpful to keep all the elements in each section alphabetized so that I can check for duplicates. == Use of element files == The prefix/suffix form is generally the most useful. Most of the example files are in this form. Note that you are not restricted to fantasy names. By using english words you can get things like "Darkwood", "Brightblade", "Millennium Falcon", "Nottingham" and so forth. See the sample files for ideas. Middles can add a lot of variety to your names, but they also make the names themselves longer. Noun/NADJ/ADJ elements are mostly useful for idea-generators, or special places like "House of the Skulls" or "The Books of the Priest." The program combines these in a variety of ways, and also pluralizes them sometimes. All you need to do is supply the list of words. Such lists may represent themes, such as dark fantasy, science fiction, nautical things, wizardly things, and so forth. Again, see the sample ELE files. It is easy to make your own ELE files. There are several methods I have found personally useful. One is to just sit down and starting making up the elements out of your head. Just come up with all the combinations you can think of and put them in. Another is to compile a list of names from your favorite book(s), movie(s), etc. You can then easily split these names into prefix/suffix parts. Example: {| ! Original name !! PRE !! SUF |- | Alcamtar || alcam || tar |- | Celowin || cel || owin |- | Conan || con || an |- | Gimli || gim || li |- | Talward || tal || ward |} One helpful strategy in making your names sound better is to end the prefixes and begin the suffixes consistently. Either end the prefixes with a vowel and begin the suffixes with a consonant, or end the prefixes with a consonant and begin the suffixes with a vowel. I generally prefer the latter. Example: {| ! !! Method #1 !! Method #2 |- | Alcamtar || alca / mtar || alcam / tar |- | Celowin || celo / win || cel / owin |- | Conan || co / nan || con / an |- | Gimli || gi / mli || gim / li |- | Talward || talwa / rd || tal / ward |} This technique will keep your names from sounding too random, and will make them more pronounceable. Note that soft sounds such as r, l, t, s, w, y might be including with the vowels, as I have done above. Use your imagination and common sense, and put a few together yourself to see how they sound. Another helpful technique is to make all the words of a given language use the same set of sounds, or same spelling. This might require a little research. Some good references are the appendices to the Lord of the Rings, or an introductory book on phonetics/language from the library. Some simple examples: {| | Elvish spelling: || 'C' || 'G' || 'DH' || 'T' || 'S' || 'W' || 'F' |- | Orcish spelling: || 'K' || 'GH' || 'J' || 'D' || 'Z' || 'V' || 'V' |} Here is a chart. I'm not going to take the time to try an explain it, but study it and compare the sounds to one another. You will notice patterns. It probably isn't too accurate, but is still useful. (Yes, I got the idea from Tolkein.) {| ! !!colspan=2| STOPS !!colspan=2| SPIRANTS !!colspan=2| OTHER !!colspan=2| NASAL |- ! !! Soft !! Voiced !! Soft !! Voiced !! Soft !! Voiced !! Soft !! Voiced |- ! Labial |p || b || || bh || wh || w || mh || m |- ! Labiodental | || || f || v || || y || || |- ! Dental | || || th || th || || || || |- ! Palatal? | t || d || ch || j || lh || l || nh || n |- ! Sibilant | s || z || sh || zh || rh || r || || |- ! Alveolar? | c,q,k || g || ch || gh || h || h || || ng |} You can use this to create "sounds" for languages. For instance, celtic has a lot of unvoiced (soft) sounds, making it sound smooth and soft and flowing. German, on the other hand, has a lot of voiced stops, making it louder and harsher sounding. By selecting which sounds are most common, you can give each fantasy language a certain "feel." One handy technique is to get an atlas of foreign countries and look at the names on the map. You can get a pretty good idea of which sounds are common, and how they should go together this way. This is the way I did my "french" ELE file, and also the "aztec" and "african" ones. == Conclusion == Hope you find this useful. I can be contacted for comments/questions on the Internet as "mike@cs.pdx.edu". [[Category:Name Generation]] c621eda346d32ea37a72ae2703c1eca301adf956 1819 1817 2008-09-06T21:30:12Z Neil 2 wikitext text/x-wiki ''(This was written by Michael Harvey to support his [[:Category:Name Generation|name generation]] program.)'' Made-up names, especially those found in literature and roleplaying games, often sound silly, out of place, or not-quite right. To be honest, it can be difficult thinking up creative names that sound right. Names is a program which can generate hundreds of names based on the input you feed it. Because it combines "elements" from data files to form names, it is possible to generate very high quality output. Further, it is possible to create data files which provide a "feel" for a particular language or culture -- real or imaginary. This enables the writer or game designer to generate lists of names for each fantasy culture, which all have a certain sound and seem to belong together. This document has various suggestions on how to create element files, as well as how to generate good quality names. == Names elements == There are several types of name elements: ;Prefix (PRE) :Parts that begin a word. Example: "c", "co", and "con" could all be prefixes for "Conan" ;Middle (MID) :Parts that go in the middle of a word, ''between'' a prefix and suffix. Example: "cam" is a middle for "Alcamtar", where "al" is the prefix and "tar" is the suffix ;Suffix (SUF) :Parts that end a word. In the name Aragorn, suffixes could be "agorn", "gorn", "orn", or even "n" ;Noun (NOUN) :Nouns: "road", "fortress", "tree", "crossing" ;Adjective (ADJ) :Adjectives: "gray", "slow", "stubborn" ;Noun/adjective (NADJ) :These are words which could be either nouns or adjectives. Examples: dark ("darkwood", "outer dark"), silver ("silverlode", "moria-silver"), wood ("woodtown", "goblin-wood") :Many nouns and adjectives actually fall into this category. This gives the program more flexibility in using them, providing much more variety in names. Elements should not be duplicated in two categories. For example, "orc" should not be under both noun and adjective, or you might get something like "Orcorc". Instead, make it a NADJ and it won't get used twice. Elements that you want to emphasize may be duplicated within the same category. For instance, if you list "forest" three times in a row, it is three times more likely to be selected. Elements are stored in plain text .ELE files. Thus, you might have one file of goblin sounding names and another for elvish words, etc. == Element files == These should have the extension <code>.ELE</code>. The element file contains several types of elements, denoted by simple abbreviations: {| |<code>PRE</code>||prefixes |- |<code>MID</code>||middles |- |<code>SUF</code>||suffixes |- |<code>NOUN</code>||nouns |- |<code>ADJ</code>||adjectives |- |<code>NADJ</code>||noun/adjectives |- |<code>END</code>||this signifies the end of the file |} The pound sign (#) is used as an escape character to denote the start of a new section (much like preprocessor directives in C). Thus, a typical file would look like this: #PRE con gal tar #SUF an arok #END Example output: Conan Galan Taran Galarok Conarok Tararok This defines three prefixes (con,gal,tar) and two suffixes (an,arok). Notice the #END used to end the file. There should not be any blank lines in the file, unless you want to define empty parts. Parts may include any character, including spaces. For example, #PRE con gal tar #SUF an 'Arok -Rog e arba zud #END Might produce the following output: Con An Gal'Arok Galzud Tar-Rog Cone Arba Elements in an element file should not be capitalized. The program will automatically capitalize the names after each space. If you do put any capitals in, they will not be converted to lowercase, so this could be used to artificially capitalize words (usually after special punctuation, as above). The exception to this general "rule" is when you want to capitalize an element after a punctuation mark; in the example above, 'Arok would not have been automatically capitalized because there is no whitespace. I find it helpful to keep all the elements in each section alphabetized so that I can check for duplicates. == Use of element files == The prefix/suffix form is generally the most useful. Most of the example files are in this form. Note that you are not restricted to fantasy names. By using english words you can get things like "Darkwood", "Brightblade", "Millennium Falcon", "Nottingham" and so forth. See the sample files for ideas. Middles can add a lot of variety to your names, but they also make the names themselves longer. Noun/NADJ/ADJ elements are mostly useful for idea-generators, or special places like "House of the Skulls" or "The Books of the Priest." The program combines these in a variety of ways, and also pluralizes them sometimes. All you need to do is supply the list of words. Such lists may represent themes, such as dark fantasy, science fiction, nautical things, wizardly things, and so forth. Again, see the sample ELE files. It is easy to make your own ELE files. There are several methods I have found personally useful. One is to just sit down and starting making up the elements out of your head. Just come up with all the combinations you can think of and put them in. Another is to compile a list of names from your favorite book(s), movie(s), etc. You can then easily split these names into prefix/suffix parts. Example: {| ! Original name !! PRE !! SUF |- | Alcamtar || alcam || tar |- | Celowin || cel || owin |- | Conan || con || an |- | Gimli || gim || li |- | Talward || tal || ward |} One helpful strategy in making your names sound better is to end the prefixes and begin the suffixes consistently. Either end the prefixes with a vowel and begin the suffixes with a consonant, or end the prefixes with a consonant and begin the suffixes with a vowel. I generally prefer the latter. Example: {| ! !! Method #1 !! Method #2 |- | Alcamtar || alca / mtar || alcam / tar |- | Celowin || celo / win || cel / owin |- | Conan || co / nan || con / an |- | Gimli || gi / mli || gim / li |- | Talward || talwa / rd || tal / ward |} This technique will keep your names from sounding too random, and will make them more pronounceable. Note that soft sounds such as r, l, t, s, w, y might be including with the vowels, as I have done above. Use your imagination and common sense, and put a few together yourself to see how they sound. Another helpful technique is to make all the words of a given language use the same set of sounds, or same spelling. This might require a little research. Some good references are the appendices to the Lord of the Rings, or an introductory book on phonetics/language from the library. Some simple examples: {| | Elvish spelling: || 'C' || 'G' || 'DH' || 'T' || 'S' || 'W' || 'F' |- | Orcish spelling: || 'K' || 'GH' || 'J' || 'D' || 'Z' || 'V' || 'V' |} Here is a chart. I'm not going to take the time to try an explain it, but study it and compare the sounds to one another. You will notice patterns. It probably isn't too accurate, but is still useful. (Yes, I got the idea from Tolkein.) {| ! !!colspan=2| STOPS !!colspan=2| SPIRANTS !!colspan=2| OTHER !!colspan=2| NASAL |- ! !! Soft !! Voiced !! Soft !! Voiced !! Soft !! Voiced !! Soft !! Voiced |- ! Labial |p || b || || bh || wh || w || mh || m |- ! Labiodental | || || f || v || || y || || |- ! Dental | || || th || th || || || || |- ! Palatal? | t || d || ch || j || lh || l || nh || n |- ! Sibilant | s || z || sh || zh || rh || r || || |- ! Alveolar? | c,q,k || g || ch || gh || h || h || || ng |} You can use this to create "sounds" for languages. For instance, celtic has a lot of unvoiced (soft) sounds, making it sound smooth and soft and flowing. German, on the other hand, has a lot of voiced stops, making it louder and harsher sounding. By selecting which sounds are most common, you can give each fantasy language a certain "feel." One handy technique is to get an atlas of foreign countries and look at the names on the map. You can get a pretty good idea of which sounds are common, and how they should go together this way. This is the way I did my "french" ELE file, and also the "aztec" and "african" ones. == Conclusion == Hope you find this useful. I can be contacted for comments/questions on the Internet as "mike@cs.pdx.edu". [[Category:Name generation]] fe02fe8589c418d0ed9eb485c244c1d42da93db4 14 1550 1820 2008-09-06T21:30:45Z Neil 2 New page: I've done some investigating into the various names used by different cultures in and around Prax. The intention was that players should have an idea of a character's culture from his or h... wikitext text/x-wiki I've done some investigating into the various names used by different cultures in and around Prax. The intention was that players should have an idea of a character's culture from his or her name. There are two main ways of doing this: the syllable-merging approach, and the ''n''-gram model approach. I've written a little program to to both. == Syllable Merging == Michael Harvey developed a program that created random words and names from a set of syllables (the [http://scripts.njae.me.uk/downloads/name-generation/names201.zip original zip file] is available). I translated the program to Delphi to give it a nicer user interface: hopefully it's intuitive to use!. [ [http://scripts.njae.me.uk/downloads/name-generation/namegen.zip Zipped executable] (runs under Win3.1 and later) | [http://scripts.njae.me.uk/downloads/name-generation/namegen-source.zip Source] | [[How syllable merging works|How it works]] (Michael's original document) ] Here are some example name element files (the raw outputs given need judgment before use): * [http://scripts.njae.me.uk/downloads/name-generation/prax.ele Praxian] (use "Enhanced noun creation") [ [http://scripts.njae.me.uk/downloads/name-generation/prax.txt sample output] ] * Sartarite [http://scripts.njae.me.uk/downloads/name-generation/sartar-male.ele male names] and [http://scripts.njae.me.uk/downloads/name-generation/sartar-male-epithet.ele epithets], [http://scripts.njae.me.uk/downloads/name-generation/sartar-female.ele female names] and [http://scripts.njae.me.uk/downloads/name-generation/sartar-male-epithet.ele epithets] (based on Old English and Viking epithets) [ [http://scripts.njae.me.uk/downloads/name-generation/sartar-male.txt sample male output] | [http://scripts.njae.me.uk/downloads/name-generation/sartar-female.txt sample female output] ] * Lunar [http://scripts.njae.me.uk/downloads/name-generation/lunar-male.ele male] and [http://scripts.njae.me.uk/downloads/name-generation/lunar-female.ele female] names (based on Latin: use male names as surnames) [ [http://scripts.njae.me.uk/downloads/name-generation/lunar-male.txt sample male output] | [http://scripts.njae.me.uk/downloads/name-generation/lunar-female.txt sample female output] ] * [http://scripts.njae.me.uk/downloads/name-generation/dara-happa.ele Dara Happan] (Sumerian / Assyrian) [ [http://scripts.njae.me.uk/downloads/name-generation/dara-happa.txt sample output] ] * [http://scripts.njae.me.uk/downloads/name-generation/pavis.ele Pavic] (based on Gaelic) [ [http://scripts.njae.me.uk/downloads/name-generation/pavis.txt sample output] ] * Yelmalion [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-male.ele male] and [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-female.ele female] names (based on Basque) and [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-surname.ele family names] [ [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-male.txt sample male output] | [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-female.txt sample female output] ] * [http://scripts.njae.me.uk/downloads/name-generation/oasis.ele Oasis people] (based on Inca) [ [http://scripts.njae.me.uk/downloads/name-generation/oasis.txt sample output] ] * [http://scripts.njae.me.uk/downloads/name-generation/agamor.ele Agamor] (vaguely African) [ [http://scripts.njae.me.uk/downloads/name-generation/agamor.txt sample output] ] * [http://scripts.njae.me.uk/downloads/name-generation/baboon.ele Baboon] [ [http://scripts.njae.me.uk/downloads/name-generation/baboon.txt sample output] ] * [http://scripts.njae.me.uk/downloads/name-generation/boat.ele Boat People] (based on hobbit names!) [ [http://scripts.njae.me.uk/downloads/name-generation/boat.txt sample output] ] == ''N''-gram model approach == The other way to do this is to build a 'model' of the language. The idea is that the preceeding few letters in a word determine what the next letter could be. Let's say we're looking at bigrams, sequences of two letters (n = 2). If we take all the words in the language sample we've got, we can list all the bigrams that occur in all the words. We can also list, for each bigram, the letter that comes after it. We also record 'end of word' as being a possible successor letter for a bigram. We end up with a list of all the bigrams in the language sample, how frequent they are, and what letter follows. We also keep a list of the initial bigrams, so we know how words are allowed to start. This is our model of the language. To generate new words, we pick a random starting bigram from the list of initial bigrams. This gives us the first two letters of our word. We then look up that bigram in our main list of bigrams, which gives us a list of letters that can follow this bigram. We pick one of those at random, and that gives us the third letter of our word. We then take the bigram of the second and third letters and look it up in the list of bigrams; from this, we generate the fourth letter. We then use the third and fourth letter to generate the fifth, and so on until we choose an 'end of word' marker. Using larger values for n means that the generated words conform more closely to the words in the language sample, but there is a tendency to recycle the existing words if the sample is small. I find that using trigrams (n = 3) works well when there's a few hundred words. [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramords] ([[ngramwords documentation|documentation]]) is a [http://www.tcl.tk Tcl] script that creates an n-gram model of a language and then generates new random words based on that model. It's released under the [http://scripts.njae.me.uk/downloads/gpl.txt GNU Public Licence (GPL)], meaning that it is free and open-source software. I've mainly used this program to generate words for [http://www.tekumel.com Tekumel]-based games (mainly Tsolyani ones). The [http://scripts.njae.me.uk/downloads/name-generation/tekumel-source-words.txt word list I use] is based on one posted to the [http://games.groups.yahoo.com/group/tekumel/ Tekumel mailing list] and name lists at the [http://www.tekumel.com Tekumel website]. Here are [http://scripts.njae.me.uk/downloads/name-generation/tekumel-generated-words.txt some sample words]. [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramords] is based heavily on <code>lc</code>, one of [http://www.ruf.rice.edu/~pound/ Chris Pound's name generation scripts], but with a couple of extensions. * First, ngramords treats transliterated ligatures as single characters. Some languages, like Greek, have single letters (like theta and phi) that are transliterated into English as more than one letter ('th' and 'ph'). If these 'ligatures' are treated as separate letters, the language model loses information as the component parts of the ligature take up slots in the n-gram. If ngramords is told about these ligatures, it will treat them as single tokens. * Second, if lc generates a word that is too long, it just returns what it's generated so far. This may not be a valid word if there are rules in the language for what the ends of words are like (for instance, all Japanese words end in 'n' or a vowel). In contrast, if ngramords generates a word that is too long, it abandons that word and starts again. [[Category:RPGs]] 7c6efc1bd96f16688226b293016ac50967345d4e 1826 1820 2008-09-06T21:51:56Z Neil 2 /* ''N''-gram model approach */ wikitext text/x-wiki I've done some investigating into the various names used by different cultures in and around Prax. The intention was that players should have an idea of a character's culture from his or her name. There are two main ways of doing this: the syllable-merging approach, and the ''n''-gram model approach. I've written a little program to to both. == Syllable Merging == Michael Harvey developed a program that created random words and names from a set of syllables (the [http://scripts.njae.me.uk/downloads/name-generation/names201.zip original zip file] is available). I translated the program to Delphi to give it a nicer user interface: hopefully it's intuitive to use!. [ [http://scripts.njae.me.uk/downloads/name-generation/namegen.zip Zipped executable] (runs under Win3.1 and later) | [http://scripts.njae.me.uk/downloads/name-generation/namegen-source.zip Source] | [[How syllable merging works|How it works]] (Michael's original document) ] Here are some example name element files (the raw outputs given need judgment before use): * [http://scripts.njae.me.uk/downloads/name-generation/prax.ele Praxian] (use "Enhanced noun creation") [ [http://scripts.njae.me.uk/downloads/name-generation/prax.txt sample output] ] * Sartarite [http://scripts.njae.me.uk/downloads/name-generation/sartar-male.ele male names] and [http://scripts.njae.me.uk/downloads/name-generation/sartar-male-epithet.ele epithets], [http://scripts.njae.me.uk/downloads/name-generation/sartar-female.ele female names] and [http://scripts.njae.me.uk/downloads/name-generation/sartar-male-epithet.ele epithets] (based on Old English and Viking epithets) [ [http://scripts.njae.me.uk/downloads/name-generation/sartar-male.txt sample male output] | [http://scripts.njae.me.uk/downloads/name-generation/sartar-female.txt sample female output] ] * Lunar [http://scripts.njae.me.uk/downloads/name-generation/lunar-male.ele male] and [http://scripts.njae.me.uk/downloads/name-generation/lunar-female.ele female] names (based on Latin: use male names as surnames) [ [http://scripts.njae.me.uk/downloads/name-generation/lunar-male.txt sample male output] | [http://scripts.njae.me.uk/downloads/name-generation/lunar-female.txt sample female output] ] * [http://scripts.njae.me.uk/downloads/name-generation/dara-happa.ele Dara Happan] (Sumerian / Assyrian) [ [http://scripts.njae.me.uk/downloads/name-generation/dara-happa.txt sample output] ] * [http://scripts.njae.me.uk/downloads/name-generation/pavis.ele Pavic] (based on Gaelic) [ [http://scripts.njae.me.uk/downloads/name-generation/pavis.txt sample output] ] * Yelmalion [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-male.ele male] and [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-female.ele female] names (based on Basque) and [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-surname.ele family names] [ [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-male.txt sample male output] | [http://scripts.njae.me.uk/downloads/name-generation/yelmalio-female.txt sample female output] ] * [http://scripts.njae.me.uk/downloads/name-generation/oasis.ele Oasis people] (based on Inca) [ [http://scripts.njae.me.uk/downloads/name-generation/oasis.txt sample output] ] * [http://scripts.njae.me.uk/downloads/name-generation/agamor.ele Agamor] (vaguely African) [ [http://scripts.njae.me.uk/downloads/name-generation/agamor.txt sample output] ] * [http://scripts.njae.me.uk/downloads/name-generation/baboon.ele Baboon] [ [http://scripts.njae.me.uk/downloads/name-generation/baboon.txt sample output] ] * [http://scripts.njae.me.uk/downloads/name-generation/boat.ele Boat People] (based on hobbit names!) [ [http://scripts.njae.me.uk/downloads/name-generation/boat.txt sample output] ] == ''N''-gram model approach == The other way to do this is to build a 'model' of the language. The idea is that the preceeding few letters in a word determine what the next letter could be. Let's say we're looking at bigrams, sequences of two letters (''n'' = 2). If we take all the words in the language sample we've got, we can list all the bigrams that occur in all the words. We can also list, for each bigram, the letter that comes after it. We also record 'end of word' as being a possible successor letter for a bigram. We end up with a list of all the bigrams in the language sample, how frequent they are, and what letter follows. We also keep a list of the initial bigrams, so we know how words are allowed to start. This is our model of the language. To generate new words, we pick a random starting bigram from the list of initial bigrams. This gives us the first two letters of our word. We then look up that bigram in our main list of bigrams, which gives us a list of letters that can follow this bigram. We pick one of those at random, and that gives us the third letter of our word. We then take the bigram of the second and third letters and look it up in the list of bigrams; from this, we generate the fourth letter. We then use the third and fourth letter to generate the fifth, and so on until we choose an 'end of word' marker. Using larger values for n means that the generated words conform more closely to the words in the language sample, but there is a tendency to recycle the existing words if the sample is small. I find that using trigrams (''n'' = 3) works well when there's a few hundred words. [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramords] ([[ngramwords documentation|documentation]]) is a [http://www.tcl.tk Tcl] script that creates an n-gram model of a language and then generates new random words based on that model. It's released under the [http://scripts.njae.me.uk/downloads/gpl.txt GNU Public Licence (GPL)], meaning that it is free and open-source software. I've mainly used this program to generate words for [http://www.tekumel.com Tekumel]-based games (mainly Tsolyani ones). The [http://scripts.njae.me.uk/downloads/name-generation/tekumel-source-words.txt word list I use] is based on one posted to the [http://games.groups.yahoo.com/group/tekumel/ Tekumel mailing list] and name lists at the [http://www.tekumel.com Tekumel website]. Here are [http://scripts.njae.me.uk/downloads/name-generation/tekumel-generated-words.txt some sample words]. [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramords] is based heavily on <code>lc</code>, one of [http://www.ruf.rice.edu/~pound/ Chris Pound's name generation scripts], but with a couple of extensions. * First, ngramords treats transliterated ligatures as single characters. Some languages, like Greek, have single letters (like theta and phi) that are transliterated into English as more than one letter ('th' and 'ph'). If these 'ligatures' are treated as separate letters, the language model loses information as the component parts of the ligature take up slots in the n-gram. If ngramords is told about these ligatures, it will treat them as single tokens. * Second, if lc generates a word that is too long, it just returns what it's generated so far. This may not be a valid word if there are rules in the language for what the ends of words are like (for instance, all Japanese words end in 'n' or a vowel). In contrast, if ngramords generates a word that is too long, it abandons that word and starts again. [[Category:RPGs]] d44bd6b93a3b76a48e06ade37485245e67765719 0 1551 1821 2008-09-06T21:35:48Z Neil 2 New page: Version 1.0, 8 August 2005 Written by and copyright Neil Smith [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is a simple Tcl program that generates random wo... wikitext text/x-wiki Version 1.0, 8 August 2005 Written by and copyright Neil Smith [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is a simple Tcl program that generates random words based on a sample text. == Introduction == The program reads a set of words and generates an n-gram model of the language. It then uses that model to generate new random words. The idea is that the preceeding few letters in a word determine what the next letter could be. Let's say we're looking at bigrams, sequences of two letters (n = 2). If we take all the words in the language sample we've got, we can list all the bigrams that occur in all the words. We can also list, for each bigram, the letter that comes after it. We also record 'end of word' as being a possible successor letter for a bigram. We end up with a list of all the bigrams in the language sample, how frequent they are, and what letter follows. We also keep a list of the initial bigrams, so we know how words are allowed to start. This is our model of the language. To generate new words, we pick a random starting bigram from the list of initial bigrams. This gives us the first two letters of our word. We then look up that bigram in our main list of bigrams, which gives us a list of letters that can follow this bigram. We pick one of those at random, and that gives us the third letter of our word. We then take the bigram of the second and third letters and look it up in the list of bigrams; from this, we generate the fourth letter. We then use the third and fourth letter to generate the fifth, and so on until we choose an 'end of word' marker. Using larger values for n means that the generated words conform more closely to the words in the language sample, but there is a tendency to recycle the exising words if the sample is small. I find that using trigrams n = 3) works well when there's a few hundred words. == Installation and Use == [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is written in [http://www.tcl.tk/ Tcl], so you'll need to have a [http://www.tcl.tk/software/tcltk/ Tcl interpreter] installed and configured correctly before it will work. Copy the ngramwords file to your computer, preferably on your path. Make the file executable. The program is command-line only, so you'll neet to run a terminal to use it. To invoke the program, call it with: ngramwords [options] input-file where input-file is a text file containing a list of words to build the language model from. Words must be separated by whitespace. If any words contain any characters outside the range [A-Z]|[a-z], including accented characters, those characters must be listed in the ligatures line. If no input file name is given, ngramwords will read words from standard input. Generated words will be sent to standard output. Options ngramwords accepts several command-line options. They are: * -g n : Uses n as length of n-grams. Default 3 * -n n : Generates n words. Default 20 * -l n : Words are at most n letters long. Default 15 * -i f : Reads input from file f. Default is standard input * -o f : Writes output to file f. Default is standard output * -s : Show dictionary contents * -c : case sensitive (defult is to convert all letters to lowercase) * -v : Verbose mode === Ligatures and accented characters === Many languages, when transliterated in to the Latin alphabet, use more than one Latin letter for each native letter. To accommodate this, this program allows each element of the n-gram to be a multi-letter 'token.' This is done by including in the input file a line such as: ligatures= th ch ú (this must be on a separate line in the input file). This will mean that 'ú' will be recognized as a valid character in this language, and that 'th' and 'ch' will be treated as a single letter. See the sample input file for an example. Lines in the input file that start with a # character are treated as comments and ignored by the program. [[Category:Name generation]] 1509c29a22a153689fe3edfb067877829392f57e 1824 1821 2008-09-06T21:45:44Z Neil 2 wikitext text/x-wiki Version 1.0, 8 August 2005 Written by and copyright Neil Smith Available for download and use under the [http://scripts.njae.me.uk/downloads/gpl.txt GNU Public Licence (GPL)]. [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is a simple Tcl program that generates random words based on a sample text. == Introduction == The program reads a set of words and generates an n-gram model of the language. It then uses that model to generate new random words. The idea is that the preceeding few letters in a word determine what the next letter could be. Let's say we're looking at bigrams, sequences of two letters (n = 2). If we take all the words in the language sample we've got, we can list all the bigrams that occur in all the words. We can also list, for each bigram, the letter that comes after it. We also record 'end of word' as being a possible successor letter for a bigram. We end up with a list of all the bigrams in the language sample, how frequent they are, and what letter follows. We also keep a list of the initial bigrams, so we know how words are allowed to start. This is our model of the language. To generate new words, we pick a random starting bigram from the list of initial bigrams. This gives us the first two letters of our word. We then look up that bigram in our main list of bigrams, which gives us a list of letters that can follow this bigram. We pick one of those at random, and that gives us the third letter of our word. We then take the bigram of the second and third letters and look it up in the list of bigrams; from this, we generate the fourth letter. We then use the third and fourth letter to generate the fifth, and so on until we choose an 'end of word' marker. Using larger values for n means that the generated words conform more closely to the words in the language sample, but there is a tendency to recycle the exising words if the sample is small. I find that using trigrams n = 3) works well when there's a few hundred words. == Installation and Use == [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is written in [http://www.tcl.tk/ Tcl], so you'll need to have a [http://www.tcl.tk/software/tcltk/ Tcl interpreter] installed and configured correctly before it will work. Copy the ngramwords file to your computer, preferably on your path. Make the file executable. The program is command-line only, so you'll neet to run a terminal to use it. To invoke the program, call it with: ngramwords [options] input-file where input-file is a text file containing a list of words to build the language model from. Words must be separated by whitespace. If any words contain any characters outside the range [A-Z]|[a-z], including accented characters, those characters must be listed in the ligatures line. If no input file name is given, ngramwords will read words from standard input. Generated words will be sent to standard output. Options ngramwords accepts several command-line options. They are: * -g n : Uses n as length of n-grams. Default 3 * -n n : Generates n words. Default 20 * -l n : Words are at most n letters long. Default 15 * -i f : Reads input from file f. Default is standard input * -o f : Writes output to file f. Default is standard output * -s : Show dictionary contents * -c : case sensitive (defult is to convert all letters to lowercase) * -v : Verbose mode === Ligatures and accented characters === Many languages, when transliterated in to the Latin alphabet, use more than one Latin letter for each native letter. To accommodate this, this program allows each element of the n-gram to be a multi-letter 'token.' This is done by including in the input file a line such as: ligatures= th ch ú (this must be on a separate line in the input file). This will mean that 'ú' will be recognized as a valid character in this language, and that 'th' and 'ch' will be treated as a single letter. See the sample input file for an example. Lines in the input file that start with a # character are treated as comments and ignored by the program. [[Category:Name generation]] 179feac7a29fdd4a604aa2648fb4b815204082c8 1825 1824 2008-09-06T21:51:44Z Neil 2 wikitext text/x-wiki Version 1.0, 8 August 2005 Written by and copyright Neil Smith Available for download and use under the [http://scripts.njae.me.uk/downloads/gpl.txt GNU Public Licence (GPL)]. [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is a simple Tcl program that generates random words based on a sample text. == Introduction == The program reads a set of words and generates an n-gram model of the language. It then uses that model to generate new random words. The idea is that the preceeding few letters in a word determine what the next letter could be. Let's say we're looking at bigrams, sequences of two letters (''n'' = 2). If we take all the words in the language sample we've got, we can list all the bigrams that occur in all the words. We can also list, for each bigram, the letter that comes after it. We also record 'end of word' as being a possible successor letter for a bigram. We end up with a list of all the bigrams in the language sample, how frequent they are, and what letter follows. We also keep a list of the initial bigrams, so we know how words are allowed to start. This is our model of the language. To generate new words, we pick a random starting bigram from the list of initial bigrams. This gives us the first two letters of our word. We then look up that bigram in our main list of bigrams, which gives us a list of letters that can follow this bigram. We pick one of those at random, and that gives us the third letter of our word. We then take the bigram of the second and third letters and look it up in the list of bigrams; from this, we generate the fourth letter. We then use the third and fourth letter to generate the fifth, and so on until we choose an 'end of word' marker. Using larger values for n means that the generated words conform more closely to the words in the language sample, but there is a tendency to recycle the exising words if the sample is small. I find that using trigrams (''n'' = 3) works well when there's a few hundred words. == Installation and Use == [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is written in [http://www.tcl.tk/ Tcl], so you'll need to have a [http://www.tcl.tk/software/tcltk/ Tcl interpreter] installed and configured correctly before it will work. Copy the ngramwords file to your computer, preferably on your path. Make the file executable. The program is command-line only, so you'll neet to run a terminal to use it. To invoke the program, call it with: ngramwords [options] input-file where input-file is a text file containing a list of words to build the language model from. Words must be separated by whitespace. If any words contain any characters outside the range [A-Z]|[a-z], including accented characters, those characters must be listed in the ligatures line. If no input file name is given, ngramwords will read words from standard input. Generated words will be sent to standard output. Options ngramwords accepts several command-line options. They are: * -g n : Uses n as length of n-grams. Default 3 * -n n : Generates n words. Default 20 * -l n : Words are at most n letters long. Default 15 * -i f : Reads input from file f. Default is standard input * -o f : Writes output to file f. Default is standard output * -s : Show dictionary contents * -c : case sensitive (defult is to convert all letters to lowercase) * -v : Verbose mode === Ligatures and accented characters === Many languages, when transliterated in to the Latin alphabet, use more than one Latin letter for each native letter. To accommodate this, this program allows each element of the n-gram to be a multi-letter 'token.' This is done by including in the input file a line such as: ligatures= th ch ú (this must be on a separate line in the input file). This will mean that 'ú' will be recognized as a valid character in this language, and that 'th' and 'ch' will be treated as a single letter. See the sample input file for an example. Lines in the input file that start with a # character are treated as comments and ignored by the program. [[Category:Name generation]] c0c395a592bb8734248e470d2c63d5e945384117 1827 1825 2008-09-06T21:52:10Z Neil 2 /* Introduction */ wikitext text/x-wiki Version 1.0, 8 August 2005 Written by and copyright Neil Smith Available for download and use under the [http://scripts.njae.me.uk/downloads/gpl.txt GNU Public Licence (GPL)]. [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is a simple Tcl program that generates random words based on a sample text. == Introduction == The program reads a set of words and generates an ''n''-gram model of the language. It then uses that model to generate new random words. The idea is that the preceeding few letters in a word determine what the next letter could be. Let's say we're looking at bigrams, sequences of two letters (''n'' = 2). If we take all the words in the language sample we've got, we can list all the bigrams that occur in all the words. We can also list, for each bigram, the letter that comes after it. We also record 'end of word' as being a possible successor letter for a bigram. We end up with a list of all the bigrams in the language sample, how frequent they are, and what letter follows. We also keep a list of the initial bigrams, so we know how words are allowed to start. This is our model of the language. To generate new words, we pick a random starting bigram from the list of initial bigrams. This gives us the first two letters of our word. We then look up that bigram in our main list of bigrams, which gives us a list of letters that can follow this bigram. We pick one of those at random, and that gives us the third letter of our word. We then take the bigram of the second and third letters and look it up in the list of bigrams; from this, we generate the fourth letter. We then use the third and fourth letter to generate the fifth, and so on until we choose an 'end of word' marker. Using larger values for n means that the generated words conform more closely to the words in the language sample, but there is a tendency to recycle the exising words if the sample is small. I find that using trigrams (''n'' = 3) works well when there's a few hundred words. == Installation and Use == [http://scripts.njae.me.uk/downloads/name-generation/ngramwords ngramwords] is written in [http://www.tcl.tk/ Tcl], so you'll need to have a [http://www.tcl.tk/software/tcltk/ Tcl interpreter] installed and configured correctly before it will work. Copy the ngramwords file to your computer, preferably on your path. Make the file executable. The program is command-line only, so you'll neet to run a terminal to use it. To invoke the program, call it with: ngramwords [options] input-file where input-file is a text file containing a list of words to build the language model from. Words must be separated by whitespace. If any words contain any characters outside the range [A-Z]|[a-z], including accented characters, those characters must be listed in the ligatures line. If no input file name is given, ngramwords will read words from standard input. Generated words will be sent to standard output. Options ngramwords accepts several command-line options. They are: * -g n : Uses n as length of n-grams. Default 3 * -n n : Generates n words. Default 20 * -l n : Words are at most n letters long. Default 15 * -i f : Reads input from file f. Default is standard input * -o f : Writes output to file f. Default is standard output * -s : Show dictionary contents * -c : case sensitive (defult is to convert all letters to lowercase) * -v : Verbose mode === Ligatures and accented characters === Many languages, when transliterated in to the Latin alphabet, use more than one Latin letter for each native letter. To accommodate this, this program allows each element of the n-gram to be a multi-letter 'token.' This is done by including in the input file a line such as: ligatures= th ch ú (this must be on a separate line in the input file). This will mean that 'ú' will be recognized as a valid character in this language, and that 'th' and 'ch' will be treated as a single letter. See the sample input file for an example. Lines in the input file that start with a # character are treated as comments and ignored by the program. [[Category:Name generation]] 282b91c417cb3a57800fc1837192283a888d8d27 0 1552 1828 2008-09-06T21:53:13Z Neil 2 New page: I have written a couple of small programs: * A Tcl/Tk script that shows [http://scripts.njae.me.uk/downloads/number numbers and words] (for teaching children). * A Tcl/Tk script that pose... wikitext text/x-wiki I have written a couple of small programs: * A Tcl/Tk script that shows [http://scripts.njae.me.uk/downloads/number numbers and words] (for teaching children). * A Tcl/Tk script that poses [http://scripts.njae.me.uk/downloads/sums simple sums] (for teaching children). If you want to change the difficulties of the levels, you'll need to edit the parameters at the top of the script. * A Delphi (Windows 95) program that [http://scripts.njae.me.uk/downloads/name-generation/namegen.zip generates random words] by combining syllables. See the [[:Category:Name generation|name generation page]] for more details. * A Tcl script that [http://scripts.njae.me.uk/downloads/name-generation/ngramwords generates random words] by building an ''n''-gram model of a language. See the [[ngramwords documentation| documentation]] and the [[:Category:Name generation|name generation page]] for more details. These programs are released under the [http://scripts.njae.me.uk/downloads/gpl.txt GNU Public Licence (GPL)], meaning that they are free and open-source software. c438a22045ca95e51c42b8c76c6c43c7ca357eb4 0 1483 1829 1626 2008-09-06T21:55:26Z Neil 2 /* Serge Klazimakoff */ wikitext text/x-wiki Gareth Hanrahan's scenario ''[http://www.gameeire.com/Scenario/bplanet.html Things to do in Dyfedd When You're Genlifted]'' is a great crime caper, but the files don't include the statistics for the pregenerated characters. Here are my versions of them. Note that: # These are slightly underpowered Exceptional characters: they have the skill packages of Exceptional characters, but the attributes of Everyday characters; # Superscripts on skill ratings show the number of dice to roll for that skill (I can't keep track of aptitudes when I'm running the game). == Serge Klazimakoff == * ''Packages'': Origin: Freezone&mdash;Wasteland; Backgrounds: Colonial, Street; Professional: Expert Crime (Gangster), Novice Technical, Novice Commerce. * ''Primary Attributes'': Build +2, Fitness +2, Agility -1, Dexterity 0, Awareness -1 (Chemical -2, Hearing -1, Intuition -1, Touch -1, Vision 0), Intellect +1, Presence -1, Will 0. * ''Secondary Attributes'': Endurance +2, Reflexes -1, Strength +2, Toughness +1. * ''Skills'': Bureaucracy 4, Law 6; Aquatics 4; Armed Combat 4&sup2;, Unarmed Combat 5&sup2;; Leadership 2; Negotiation 5&sup2;, Persuasion 5&sup2;; Culture (Cetacean) 1, Culture (Colonial) 3, Culture (Earth) 3, Culture (GEO) 1, Culture (Incorporate) 2, Culture (Native) 1, Culture (Street) 6; Handguns 5, Longarms 5; First Aid 1, Forensic Medicine 1, Psychology 1; Physics 1; Hiding 4&sup2;, Shadowing 4&sup2;, Sneaking 4&sup2;; Bribery 5&sup3;, Fast Talk 6&sup3;, Lockpicking 4&sup3;, Sleight of Hand 1&sup3;; Foraging 1, Orienteering 1, Tracking 1; Computers 2&sup2;, Damage Control 2&sup2;, Electronics 2&sup2;, Mechanics 2&sup2;, Remote Operations 3&sup2;; Driving 4&sup3;, Piloting 4&sup3;. * ''Modifications'': Improved blood oxygenation, immunological symbiote, salt tolerance, pain inhibitor. * ''Equipment'': Bodycomp, utility jumpcraft, large calibre handgun, sub-machine gun, light armour vest, reinforced armour vest. == Larry Scjelli == * ''Species'': Transhuman. * ''Packages'': Origin: Incorporate; Backgrounds: Incorporate, Wealthy; Professional: Specialist Administration, Specialist Crime, Novice Diplomacy. * ''Primary Attributes'': Build 0, Fitness +1, Agility -1, Dexterity -1, Awareness +1 (Chemical +1, Hearing 0, Intuition +2, Touch +1, Vision +1), Intellect +2, Presence +4, Will +2. * ''Secondary Attributes'': Endurance +2, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 6&sup3;, Economics 5&sup3;, Law 4&sup3;, Management 3&sup3;; Armed Combat 2, Unarmed Combat 2; Leadership 4&sup2;, Logistics 2&sup2;, Strategy 3&sup2;; Language (English, Native), Language (Arabic) 5&sup3;, Language (German) 3&sup3;, Negotiation 6&sup3;, Oration 3&sup3;, Persuasion 6&sup3;, Writing 3&sup3;; Culture (Colonial) 3&sup2;, Culture (Earth) 2&sup2;, Culture (GEO) 3&sup2;, Culture (Incorporate) 8&sup2;, Culture (Native) 3&sup2;, Culture (Street) 3&sup2;; Painting 1; Handguns 2; History 22, Politics 42; Psychology 1; Hiding 3, Shadowing 3, Sneaking 3; Bribery 3&sup2;, Fast Talk 6&sup2;, Lockpicking 2&sup2;, Sleight of Hand 1&sup2;; Computers 3; Driving 1, Piloting 3. * ''Modifications'': Bodysculpting, immunological symbiote, multiglands, neural jack. * ''Equipment'': Minisub, concealable handgun. == Li Fan Tsu == * ''Packages'': Origin: Urban; Backgrounds: GEO, Independent; Professional: Expert Law Enforcement (Undercover), Novice Crime, Novice Athletics. * ''Primary Attributes'': Build 0, Fitness +3, Agility +2, Dexterity +2, Awareness +2 (Chemical +1, Hearing +5, Intuition +2, Touch +1, Vision +3), Intellect 0, Presence 0, Will -1. * ''Secondary Attributes'': Endurance 3, Reflexes 3, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 5&sup2;, Economics 2&sup2;, Law 5&sup2;; Aquatics 3&sup2;, Sports 3&sup2;, Throwing 3&sup2;; Armed Combat 3&sup2;, Unarmed Combat 5&sup2;; Language: Interspec 1, Negotiation 3, Persuasion 6; Culture (Colonial) 2, Culture (Earth) 5, Culture (GEO) 4, Culture (Incorporate) 2, Culture (Native) 3, Culture (Street) 6; Acting 3, Photography 3; Handguns 5&sup3;, Longarms 2&sup3;; History 1, Politics 1; First Aid 5, Forensic Medicine 3, General Medicine 1, Pharmacology 1; Hiding 3&sup3;, Shadowing 5&sup3;, Sneaking 3&sup3;; Bribery 1&sup2;, Fast Talk 4&sup2;, Lockpicking 3&sup2;; Computers 3; Driving 6. * ''Modifications'': Amplified hearing, improved blood oxygenation, immunological symbiote, salt tolerance, accelerated neurons, neural jack, programmed reflexes (autoload, quick draw, defender). * ''Equipment'': Bodycomp + recorders, medical kit, diving mask + artificial gill, powerboat, diamond knife, concealable handgun, large calibre handgun, light armour vest. == Abdullah Al-Maluk == * ''Packages'': Origin: Urban; Backgrounds: Incorporate, University; Professional: Expert Espionage (Operative), Novice Crime, Novice Diplomacy. * ''Primary Attributes'': Build 0, Fitness 0, Agility 0, Dexterity +1, Awareness 0 (Chemical 0, Hearing 0, Intuition +1, Touch -1, Vision 0), Intellect +1, Presence +1, Will 0. * ''Secondary Attributes'': Endurance 0, Reflexes +1, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 7, Economics 1, Law 1, Management 1; Armed Combat 4&sup2;, Unarmed Combat 4&sup2;; Strategy 1; Language (Arabic, Native), English 8&sup3;, Spanish 3&sup3;, Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 6&sup3;, Writing 3&sup3;; Culture (Colonial) 4&sup3;, Culture (Earth) 7&sup3;, Culture (Incorporate) 5&sup3;, Culture (Street) 5&sup3;; Handguns 4, Longarms 2; History 2, Politics 3, Theology 1; Psychology 1; Hiding 5, Shadowing 5, Sneaking 5; Bribery 1&sup2;, Disguise 2&sup2;, Fast Talk 6&sup2;, Lockpicking 4&sup2;; Computers 6&sup2;, Demolitions 3&sup2;, Electronics 3&sup2;, Remote Operations 3&sup2;; Driving 2. * ''Modifications'': Implanted sensory recorder, immunological symbiote, accelerated neurons, uplink neural jack. * ''Equipment'': Superior bodycomp, pain inhibitor drug &times;2, reflex serum &times;2, ultrasonic hypo, light armour vest, diamond knife, concealable handgun, small calibre handgun. == Genevive Sian == * ''Species'': Native Aquaform (Diver). * ''Packages'': Origin: Native; Backgrounds: Native, Colonial; Professional: Specialist Survival, Specialist Military, Novice Crime. * ''Primary Attributes'': Build -2, Fitness +1, Agility +1, Dexterity 0, Awareness +1 (Chemical +1, Hearing +2, Intuition +1, Touch +1, Vision 0), Intellect -1, Presence 0, Will +1. * ''Secondary Attributes'': Endurance +2, Reflexes +1, Strength 0, Toughness 0. * ''Skills'': Aquaculture 2; Aquatics 10&sup2;, Throwing 4&sup2;; Armed Combat 4&sup2;, Unarmed Combat 4&sup2;; Leadership 2, Strategy 2, Tactics 2; Interspec 4, Negotiation 1, Persuasion 1; Culture (Cetacean) 3, Culture (Colonial) 6, Culture (GEO) 1, Culture (Military) 1, Culture (Native) 6, Culture (Street) 1; Handguns 3&sup2;, Longarms 5&sup2;; Botany 2, Zoology 2; First Aid 4; Heavy Weapons 2; Meterology 2; Hiding 4&sup3;, Shadowing 2&sup3;, Sneaking 4&sup3;; Bribery 3, Fast Talk 1, Lockpicking 1; Fishing 5&sup3;, Foraging 3&sup3;, Mountaineering 3&sup3;, Navigation 4&sup3;, Orienteering 4&sup3;, Tracking 3&sup3;; Demolitions 1, Remote Operations 2; Driving 1&sup2;, Piloting 1&sup2;, Sailing 5&sup2;. * ''Modifications'': None. * ''Equipment'': Portable navigation suite, native catamaran, diamond knife, small calibre handgun, hunting rifle + optical scope. == Marlon Clark == * ''Packages'': Origin: Mars Colony; Backgrounds: Criminal, Street; Professional: Expert Crime (Thug), Novice Law Enforcement, Novice Espionage. * ''Primary Attributes'': Build 0, Fitness +2, Agility +2, Dexterity +2, Awareness -1 (Chemical -1, Hearing -1, Intuition -2, Touch -1, Vision 0), Intellect 0, Presence -2, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes +1, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 1, Law 6; Armed Combat 4&sup2;, Unarmed Combat 6&sup2;; Strategy 2&sup2;, Tactics 3&sup2;; Negotiation 4&sup2;, Persuasion 6&sup2;; Culture (Colonial) 2, Culture (Spacer) 3, Culture (Street) 6; Handguns 6&sup2;, Longarms 5&sup2;; Politics 1; Forensic Medicine 1; Hiding 6&sup3;, Shadowing 6&sup3;, Sneaking 6&sup3;; Bribery 6&sup3;, Disguise 2&sup3;, Fast Talk 6&sup3;, Lockpicking 5&sup3; Sleight of Hand 2&sup3;; Computers 2, Damage Control 1, Electronics 2, Mechanics 1, Remote Operations 2; Driving 3, Piloting 1. * ''Modifications'': Anti-poison, accelerated neurons, neural jack, implanted computer. * ''Equipment'': Concealable handgun, large calibre handgun, stun baton. == Gorchoff Thug == * Reflexes +1, Agility +1, Dexterity 0, Awareness 0, Toughness 0, Fitness +1, Will +1, Armour 2. * Handgun 5&sup2;, DR 6. == Dyffedd Security == * Reflexes +1, Agility +1, Dexterity 0, Awareness 0, Toughness 0, Fitness +1, Will +1, Armour 3. * SMG 6&sup2;, DR 6/9. * Stun Gun 6&sup2;, DR Special. == Free Poseidon Ecoterrorists == * Reflexes 0, Agility +2, Dexterity 0, Toughness +1, Fitness +2, Will 0, Armour 2. * Hunting Rifle 3, DR 9. * Knife 3&sup3;, DR 2. == Grimm == * Reflexes +3, Agility +2, Dexterity 0, Toughness +2, Fitness +2, Will +2, Armour 3. * Custom Handgun 8&sup3;, DR 7. == Vermier == * Reflexes +2, Agility +1 Dexterity +1, Toughness +1, Fitness 0, Will +1, Armour 0. * Shotgun 5&sup2;, DR 7/10. [[Category:Blue Planet]] fb017c9ddf53cb7364970c60e798f3902f781581 1830 1829 2008-09-06T21:59:12Z Neil 2 /* Larry Scjelli */ wikitext text/x-wiki Gareth Hanrahan's scenario ''[http://www.gameeire.com/Scenario/bplanet.html Things to do in Dyfedd When You're Genlifted]'' is a great crime caper, but the files don't include the statistics for the pregenerated characters. Here are my versions of them. Note that: # These are slightly underpowered Exceptional characters: they have the skill packages of Exceptional characters, but the attributes of Everyday characters; # Superscripts on skill ratings show the number of dice to roll for that skill (I can't keep track of aptitudes when I'm running the game). == Serge Klazimakoff == * ''Packages'': Origin: Freezone&mdash;Wasteland; Backgrounds: Colonial, Street; Professional: Expert Crime (Gangster), Novice Technical, Novice Commerce. * ''Primary Attributes'': Build +2, Fitness +2, Agility -1, Dexterity 0, Awareness -1 (Chemical -2, Hearing -1, Intuition -1, Touch -1, Vision 0), Intellect +1, Presence -1, Will 0. * ''Secondary Attributes'': Endurance +2, Reflexes -1, Strength +2, Toughness +1. * ''Skills'': Bureaucracy 4, Law 6; Aquatics 4; Armed Combat 4&sup2;, Unarmed Combat 5&sup2;; Leadership 2; Negotiation 5&sup2;, Persuasion 5&sup2;; Culture (Cetacean) 1, Culture (Colonial) 3, Culture (Earth) 3, Culture (GEO) 1, Culture (Incorporate) 2, Culture (Native) 1, Culture (Street) 6; Handguns 5, Longarms 5; First Aid 1, Forensic Medicine 1, Psychology 1; Physics 1; Hiding 4&sup2;, Shadowing 4&sup2;, Sneaking 4&sup2;; Bribery 5&sup3;, Fast Talk 6&sup3;, Lockpicking 4&sup3;, Sleight of Hand 1&sup3;; Foraging 1, Orienteering 1, Tracking 1; Computers 2&sup2;, Damage Control 2&sup2;, Electronics 2&sup2;, Mechanics 2&sup2;, Remote Operations 3&sup2;; Driving 4&sup3;, Piloting 4&sup3;. * ''Modifications'': Improved blood oxygenation, immunological symbiote, salt tolerance, pain inhibitor. * ''Equipment'': Bodycomp, utility jumpcraft, large calibre handgun, sub-machine gun, light armour vest, reinforced armour vest. == Larry Scjelli == * ''Species'': Transhuman. * ''Packages'': Origin: Incorporate; Backgrounds: Incorporate, Wealthy; Professional: Specialist Administration, Specialist Crime, Novice Diplomacy. * ''Primary Attributes'': Build 0, Fitness +1, Agility -1, Dexterity -1, Awareness +1 (Chemical +1, Hearing 0, Intuition +2, Touch +1, Vision +1), Intellect +2, Presence +4, Will +2. * ''Secondary Attributes'': Endurance +2, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 6³, Economics 5³, Law 4³, Management 3³; Armed Combat 2, Unarmed Combat 2; Leadership 4&sup2;, Logistics 2², Strategy 3²; Language (English, Native), Language (Arabic) 5³, Language (German) 3³, Negotiation 6³, Oration 3³, Persuasion 6³, Writing 3³; Culture (Colonial) 3², Culture (Earth) 2², Culture (GEO) 3², Culture (Incorporate) 8², Culture (Native) 3², Culture (Street) 3²; Painting 1; Handguns 2; History 2², Politics 4²; Psychology 1; Hiding 3, Shadowing 3, Sneaking 3; Bribery 3², Fast Talk 6², Lockpicking 2², Sleight of Hand 1²; Computers 3; Driving 1, Piloting 3. * ''Modifications'': Bodysculpting, immunological symbiote, multiglands, neural jack. * ''Equipment'': Minisub, concealable handgun. == Li Fan Tsu == * ''Packages'': Origin: Urban; Backgrounds: GEO, Independent; Professional: Expert Law Enforcement (Undercover), Novice Crime, Novice Athletics. * ''Primary Attributes'': Build 0, Fitness +3, Agility +2, Dexterity +2, Awareness +2 (Chemical +1, Hearing +5, Intuition +2, Touch +1, Vision +3), Intellect 0, Presence 0, Will -1. * ''Secondary Attributes'': Endurance 3, Reflexes 3, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 5&sup2;, Economics 2&sup2;, Law 5&sup2;; Aquatics 3&sup2;, Sports 3&sup2;, Throwing 3&sup2;; Armed Combat 3&sup2;, Unarmed Combat 5&sup2;; Language: Interspec 1, Negotiation 3, Persuasion 6; Culture (Colonial) 2, Culture (Earth) 5, Culture (GEO) 4, Culture (Incorporate) 2, Culture (Native) 3, Culture (Street) 6; Acting 3, Photography 3; Handguns 5&sup3;, Longarms 2&sup3;; History 1, Politics 1; First Aid 5, Forensic Medicine 3, General Medicine 1, Pharmacology 1; Hiding 3&sup3;, Shadowing 5&sup3;, Sneaking 3&sup3;; Bribery 1&sup2;, Fast Talk 4&sup2;, Lockpicking 3&sup2;; Computers 3; Driving 6. * ''Modifications'': Amplified hearing, improved blood oxygenation, immunological symbiote, salt tolerance, accelerated neurons, neural jack, programmed reflexes (autoload, quick draw, defender). * ''Equipment'': Bodycomp + recorders, medical kit, diving mask + artificial gill, powerboat, diamond knife, concealable handgun, large calibre handgun, light armour vest. == Abdullah Al-Maluk == * ''Packages'': Origin: Urban; Backgrounds: Incorporate, University; Professional: Expert Espionage (Operative), Novice Crime, Novice Diplomacy. * ''Primary Attributes'': Build 0, Fitness 0, Agility 0, Dexterity +1, Awareness 0 (Chemical 0, Hearing 0, Intuition +1, Touch -1, Vision 0), Intellect +1, Presence +1, Will 0. * ''Secondary Attributes'': Endurance 0, Reflexes +1, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 7, Economics 1, Law 1, Management 1; Armed Combat 4&sup2;, Unarmed Combat 4&sup2;; Strategy 1; Language (Arabic, Native), English 8&sup3;, Spanish 3&sup3;, Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 6&sup3;, Writing 3&sup3;; Culture (Colonial) 4&sup3;, Culture (Earth) 7&sup3;, Culture (Incorporate) 5&sup3;, Culture (Street) 5&sup3;; Handguns 4, Longarms 2; History 2, Politics 3, Theology 1; Psychology 1; Hiding 5, Shadowing 5, Sneaking 5; Bribery 1&sup2;, Disguise 2&sup2;, Fast Talk 6&sup2;, Lockpicking 4&sup2;; Computers 6&sup2;, Demolitions 3&sup2;, Electronics 3&sup2;, Remote Operations 3&sup2;; Driving 2. * ''Modifications'': Implanted sensory recorder, immunological symbiote, accelerated neurons, uplink neural jack. * ''Equipment'': Superior bodycomp, pain inhibitor drug &times;2, reflex serum &times;2, ultrasonic hypo, light armour vest, diamond knife, concealable handgun, small calibre handgun. == Genevive Sian == * ''Species'': Native Aquaform (Diver). * ''Packages'': Origin: Native; Backgrounds: Native, Colonial; Professional: Specialist Survival, Specialist Military, Novice Crime. * ''Primary Attributes'': Build -2, Fitness +1, Agility +1, Dexterity 0, Awareness +1 (Chemical +1, Hearing +2, Intuition +1, Touch +1, Vision 0), Intellect -1, Presence 0, Will +1. * ''Secondary Attributes'': Endurance +2, Reflexes +1, Strength 0, Toughness 0. * ''Skills'': Aquaculture 2; Aquatics 10&sup2;, Throwing 4&sup2;; Armed Combat 4&sup2;, Unarmed Combat 4&sup2;; Leadership 2, Strategy 2, Tactics 2; Interspec 4, Negotiation 1, Persuasion 1; Culture (Cetacean) 3, Culture (Colonial) 6, Culture (GEO) 1, Culture (Military) 1, Culture (Native) 6, Culture (Street) 1; Handguns 3&sup2;, Longarms 5&sup2;; Botany 2, Zoology 2; First Aid 4; Heavy Weapons 2; Meterology 2; Hiding 4&sup3;, Shadowing 2&sup3;, Sneaking 4&sup3;; Bribery 3, Fast Talk 1, Lockpicking 1; Fishing 5&sup3;, Foraging 3&sup3;, Mountaineering 3&sup3;, Navigation 4&sup3;, Orienteering 4&sup3;, Tracking 3&sup3;; Demolitions 1, Remote Operations 2; Driving 1&sup2;, Piloting 1&sup2;, Sailing 5&sup2;. * ''Modifications'': None. * ''Equipment'': Portable navigation suite, native catamaran, diamond knife, small calibre handgun, hunting rifle + optical scope. == Marlon Clark == * ''Packages'': Origin: Mars Colony; Backgrounds: Criminal, Street; Professional: Expert Crime (Thug), Novice Law Enforcement, Novice Espionage. * ''Primary Attributes'': Build 0, Fitness +2, Agility +2, Dexterity +2, Awareness -1 (Chemical -1, Hearing -1, Intuition -2, Touch -1, Vision 0), Intellect 0, Presence -2, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes +1, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 1, Law 6; Armed Combat 4&sup2;, Unarmed Combat 6&sup2;; Strategy 2&sup2;, Tactics 3&sup2;; Negotiation 4&sup2;, Persuasion 6&sup2;; Culture (Colonial) 2, Culture (Spacer) 3, Culture (Street) 6; Handguns 6&sup2;, Longarms 5&sup2;; Politics 1; Forensic Medicine 1; Hiding 6&sup3;, Shadowing 6&sup3;, Sneaking 6&sup3;; Bribery 6&sup3;, Disguise 2&sup3;, Fast Talk 6&sup3;, Lockpicking 5&sup3; Sleight of Hand 2&sup3;; Computers 2, Damage Control 1, Electronics 2, Mechanics 1, Remote Operations 2; Driving 3, Piloting 1. * ''Modifications'': Anti-poison, accelerated neurons, neural jack, implanted computer. * ''Equipment'': Concealable handgun, large calibre handgun, stun baton. == Gorchoff Thug == * Reflexes +1, Agility +1, Dexterity 0, Awareness 0, Toughness 0, Fitness +1, Will +1, Armour 2. * Handgun 5&sup2;, DR 6. == Dyffedd Security == * Reflexes +1, Agility +1, Dexterity 0, Awareness 0, Toughness 0, Fitness +1, Will +1, Armour 3. * SMG 6&sup2;, DR 6/9. * Stun Gun 6&sup2;, DR Special. == Free Poseidon Ecoterrorists == * Reflexes 0, Agility +2, Dexterity 0, Toughness +1, Fitness +2, Will 0, Armour 2. * Hunting Rifle 3, DR 9. * Knife 3&sup3;, DR 2. == Grimm == * Reflexes +3, Agility +2, Dexterity 0, Toughness +2, Fitness +2, Will +2, Armour 3. * Custom Handgun 8&sup3;, DR 7. == Vermier == * Reflexes +2, Agility +1 Dexterity +1, Toughness +1, Fitness 0, Will +1, Armour 0. * Shotgun 5&sup2;, DR 7/10. [[Category:Blue Planet]] 75afc844253e3fdb9a11e9111415d2951f0087dc 1831 1830 2008-09-11T19:56:02Z Neil 2 Changed superscript markup wikitext text/x-wiki Gareth Hanrahan's scenario ''[http://www.gameeire.com/Scenario/bplanet.html Things to do in Dyfedd When You're Genlifted]'' is a great crime caper, but the files don't include the statistics for the pregenerated characters. Here are my versions of them. Note that: # These are slightly underpowered Exceptional characters: they have the skill packages of Exceptional characters, but the attributes of Everyday characters; # Superscripts on skill ratings show the number of dice to roll for that skill (I can't keep track of aptitudes when I'm running the game). == Serge Klazimakoff == * ''Packages'': Origin: Freezone&mdash;Wasteland; Backgrounds: Colonial, Street; Professional: Expert Crime (Gangster), Novice Technical, Novice Commerce. * ''Primary Attributes'': Build +2, Fitness +2, Agility -1, Dexterity 0, Awareness -1 (Chemical -2, Hearing -1, Intuition -1, Touch -1, Vision 0), Intellect +1, Presence -1, Will 0. * ''Secondary Attributes'': Endurance +2, Reflexes -1, Strength +2, Toughness +1. * ''Skills'': Bureaucracy 4, Law 6; Aquatics 4; Armed Combat 4², Unarmed Combat 5²; Leadership 2; Negotiation 5², Persuasion 5²; Culture (Cetacean) 1, Culture (Colonial) 3, Culture (Earth) 3, Culture (GEO) 1, Culture (Incorporate) 2, Culture (Native) 1, Culture (Street) 6; Handguns 5, Longarms 5; First Aid 1, Forensic Medicine 1, Psychology 1; Physics 1; Hiding 4², Shadowing 4², Sneaking 4²; Bribery 5³, Fast Talk 6³, Lockpicking 4³, Sleight of Hand 1³; Foraging 1, Orienteering 1, Tracking 1; Computers 2², Damage Control 2², Electronics 2², Mechanics 2², Remote Operations 3²; Driving 4³, Piloting 4³. * ''Modifications'': Improved blood oxygenation, immunological symbiote, salt tolerance, pain inhibitor. * ''Equipment'': Bodycomp, utility jumpcraft, large calibre handgun, sub-machine gun, light armour vest, reinforced armour vest. == Larry Scjelli == * ''Species'': Transhuman. * ''Packages'': Origin: Incorporate; Backgrounds: Incorporate, Wealthy; Professional: Specialist Administration, Specialist Crime, Novice Diplomacy. * ''Primary Attributes'': Build 0, Fitness +1, Agility -1, Dexterity -1, Awareness +1 (Chemical +1, Hearing 0, Intuition +2, Touch +1, Vision +1), Intellect +2, Presence +4, Will +2. * ''Secondary Attributes'': Endurance +2, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 6³, Economics 5³, Law 4³, Management 3³; Armed Combat 2, Unarmed Combat 2; Leadership 4², Logistics 2², Strategy 3²; Language (English, Native), Language (Arabic) 5³, Language (German) 3³, Negotiation 6³, Oration 3³, Persuasion 6³, Writing 3³; Culture (Colonial) 3², Culture (Earth) 2², Culture (GEO) 3², Culture (Incorporate) 8², Culture (Native) 3², Culture (Street) 3²; Painting 1; Handguns 2; History 2², Politics 4²; Psychology 1; Hiding 3, Shadowing 3, Sneaking 3; Bribery 3², Fast Talk 6², Lockpicking 2², Sleight of Hand 1²; Computers 3; Driving 1, Piloting 3. * ''Modifications'': Bodysculpting, immunological symbiote, multiglands, neural jack. * ''Equipment'': Minisub, concealable handgun. == Li Fan Tsu == * ''Packages'': Origin: Urban; Backgrounds: GEO, Independent; Professional: Expert Law Enforcement (Undercover), Novice Crime, Novice Athletics. * ''Primary Attributes'': Build 0, Fitness +3, Agility +2, Dexterity +2, Awareness +2 (Chemical +1, Hearing +5, Intuition +2, Touch +1, Vision +3), Intellect 0, Presence 0, Will -1. * ''Secondary Attributes'': Endurance 3, Reflexes 3, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 5², Economics 2², Law 5²; Aquatics 3², Sports 3², Throwing 3²; Armed Combat 3², Unarmed Combat 5²; Language: Interspec 1, Negotiation 3, Persuasion 6; Culture (Colonial) 2, Culture (Earth) 5, Culture (GEO) 4, Culture (Incorporate) 2, Culture (Native) 3, Culture (Street) 6; Acting 3, Photography 3; Handguns 5³, Longarms 2³; History 1, Politics 1; First Aid 5, Forensic Medicine 3, General Medicine 1, Pharmacology 1; Hiding 3³, Shadowing 5³, Sneaking 3³; Bribery 1², Fast Talk 4², Lockpicking 3²; Computers 3; Driving 6. * ''Modifications'': Amplified hearing, improved blood oxygenation, immunological symbiote, salt tolerance, accelerated neurons, neural jack, programmed reflexes (autoload, quick draw, defender). * ''Equipment'': Bodycomp + recorders, medical kit, diving mask + artificial gill, powerboat, diamond knife, concealable handgun, large calibre handgun, light armour vest. == Abdullah Al-Maluk == * ''Packages'': Origin: Urban; Backgrounds: Incorporate, University; Professional: Expert Espionage (Operative), Novice Crime, Novice Diplomacy. * ''Primary Attributes'': Build 0, Fitness 0, Agility 0, Dexterity +1, Awareness 0 (Chemical 0, Hearing 0, Intuition +1, Touch -1, Vision 0), Intellect +1, Presence +1, Will 0. * ''Secondary Attributes'': Endurance 0, Reflexes +1, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 7, Economics 1, Law 1, Management 1; Armed Combat 4², Unarmed Combat 4²; Strategy 1; Language (Arabic, Native), English 8³, Spanish 3³, Negotiation 4³, Oration 1³, Persuasion 6³, Writing 3³; Culture (Colonial) 4³, Culture (Earth) 7³, Culture (Incorporate) 5³, Culture (Street) 5³; Handguns 4, Longarms 2; History 2, Politics 3, Theology 1; Psychology 1; Hiding 5, Shadowing 5, Sneaking 5; Bribery 1², Disguise 2², Fast Talk 6², Lockpicking 4²; Computers 6², Demolitions 3², Electronics 3², Remote Operations 3²; Driving 2. * ''Modifications'': Implanted sensory recorder, immunological symbiote, accelerated neurons, uplink neural jack. * ''Equipment'': Superior bodycomp, pain inhibitor drug &times;2, reflex serum &times;2, ultrasonic hypo, light armour vest, diamond knife, concealable handgun, small calibre handgun. == Genevive Sian == * ''Species'': Native Aquaform (Diver). * ''Packages'': Origin: Native; Backgrounds: Native, Colonial; Professional: Specialist Survival, Specialist Military, Novice Crime. * ''Primary Attributes'': Build -2, Fitness +1, Agility +1, Dexterity 0, Awareness +1 (Chemical +1, Hearing +2, Intuition +1, Touch +1, Vision 0), Intellect -1, Presence 0, Will +1. * ''Secondary Attributes'': Endurance +2, Reflexes +1, Strength 0, Toughness 0. * ''Skills'': Aquaculture 2; Aquatics 10², Throwing 4²; Armed Combat 4², Unarmed Combat 4²; Leadership 2, Strategy 2, Tactics 2; Interspec 4, Negotiation 1, Persuasion 1; Culture (Cetacean) 3, Culture (Colonial) 6, Culture (GEO) 1, Culture (Military) 1, Culture (Native) 6, Culture (Street) 1; Handguns 3², Longarms 5²; Botany 2, Zoology 2; First Aid 4; Heavy Weapons 2; Meterology 2; Hiding 4³, Shadowing 2³, Sneaking 4³; Bribery 3, Fast Talk 1, Lockpicking 1; Fishing 5³, Foraging 3³, Mountaineering 3³, Navigation 4³, Orienteering 4³, Tracking 3³; Demolitions 1, Remote Operations 2; Driving 1², Piloting 1², Sailing 5². * ''Modifications'': None. * ''Equipment'': Portable navigation suite, native catamaran, diamond knife, small calibre handgun, hunting rifle + optical scope. == Marlon Clark == * ''Packages'': Origin: Mars Colony; Backgrounds: Criminal, Street; Professional: Expert Crime (Thug), Novice Law Enforcement, Novice Espionage. * ''Primary Attributes'': Build 0, Fitness +2, Agility +2, Dexterity +2, Awareness -1 (Chemical -1, Hearing -1, Intuition -2, Touch -1, Vision 0), Intellect 0, Presence -2, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes +1, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 1, Law 6; Armed Combat 4², Unarmed Combat 6²; Strategy 2², Tactics 3²; Negotiation 4², Persuasion 6²; Culture (Colonial) 2, Culture (Spacer) 3, Culture (Street) 6; Handguns 6², Longarms 5²; Politics 1; Forensic Medicine 1; Hiding 6³, Shadowing 6³, Sneaking 6³; Bribery 6³, Disguise 2³, Fast Talk 6³, Lockpicking 5³ Sleight of Hand 2³; Computers 2, Damage Control 1, Electronics 2, Mechanics 1, Remote Operations 2; Driving 3, Piloting 1. * ''Modifications'': Anti-poison, accelerated neurons, neural jack, implanted computer. * ''Equipment'': Concealable handgun, large calibre handgun, stun baton. == NPCs == === Gorchoff Thug === * Reflexes +1, Agility +1, Dexterity 0, Awareness 0, Toughness 0, Fitness +1, Will +1, Armour 2. * Handgun 5², DR 6. === Dyfedd Security === * Reflexes +1, Agility +1, Dexterity 0, Awareness 0, Toughness 0, Fitness +1, Will +1, Armour 3. * SMG 6², DR 6/9. * Stun Gun 6², DR Special. === Free Poseidon Ecoterrorists === * Reflexes 0, Agility +2, Dexterity 0, Toughness +1, Fitness +2, Will 0, Armour 2. * Hunting Rifle 3, DR 9. * Knife 3³, DR 2. === Grimm === * Reflexes +3, Agility +2, Dexterity 0, Toughness +2, Fitness +2, Will +2, Armour 3. * Custom Handgun 8³, DR 7. === Vermier === * Reflexes +2, Agility +1 Dexterity +1, Toughness +1, Fitness 0, Will +1, Armour 0. * Shotgun 5², DR 7/10. [[Category:Blue Planet]] [[Category:Scenarios]] 2da9efbe19498003ef90e83a6ac053cb370323f2 0 1533 1836 1763 2008-11-17T19:49:44Z Neil 2 Fixed link to dovecot server page wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <tt>/etc/apache2/sites-available</tt>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <tt>ServerName</tt> and <tt>DocumentRoot</tt> settings to reflect the site's name and location of files. Also note the non-default location of the <tt>cgi-bin</tt> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <tt>root</tt>, and permissions of 755. Also note that the contents of the <tt>/usr/share/doc/</tt> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 restart == Secure HTTP == I use the Secure HTTP server for my webmail. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <tt>/etc/apache2/ports.conf</tt> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <tt><nowiki>www.domain.tld</nowiki></tt> == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] [[Category:Server setup]] f48c427b960569308c51f126ac129d5debbaf5b5 0 1526 1846 1845 2008-11-19T12:38:53Z Neil 2 /* Emacs configuration */ Added SLIME config wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run SLIME, type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 58f8f3be78d8fb838faef46100d22660bfb5ec90 1847 1846 2008-11-19T12:40:03Z Neil 2 /* Emacs configuration */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] 1c2e7893c0c4cd276a64bf123f8d0f7093a74726 1848 1847 2008-11-24T09:35:35Z Neil 2 /* Normal packages */ Added MMV wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] eca21c526a39eff5189bac9a33cd4de9c3debc1a 1849 1848 2008-11-24T09:37:20Z Neil 2 /* Normal packages */ Included synaptic wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun [[Category:Server setup]] d19db51e481e5d80aece3e8f3de834f019ecc2ae 1852 1849 2008-11-24T13:49:16Z Neil 2 Added Nvidia card installation wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. [[Category:Server setup]] c6fb246d69707f1e2bfced5d1c2e38f27f45d07e 1854 1852 2008-11-26T22:50:57Z Neil 2 Added KMenu actions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true [[Category:Server setup]] fa98bd5d50b6c5b931d619b2b744e8cbfec9cf62 1855 1854 2008-11-27T13:48:04Z Neil 2 /* Add KMenu actions */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> [[Category:Server setup]] 0088bb4909d1bfb3752bd2e1c024e701e3794cae 1857 1855 2008-11-29T08:50:46Z Neil 2 /* Normal packages */ Added Imagemagick wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs22 * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> [[Category:Server setup]] 7a8a441c64be2e1840a111e88c5e5bb93368ecfe 1858 1857 2008-12-17T22:25:42Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * mplayer * mplayer-fonts * xine-ui * xine-plugin * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * ghc6 (Haskell) * ghc6-prof * haskell-mode * python * perl * ruby * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> [[Category:Server setup]] 2fba11b8af206e9fd443ab0fee838f9f085df74a 1859 1858 2008-12-17T22:53:14Z Neil 2 /* Install additional packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Netbeans * VMWare server * Google Earth * VUE * NetLogo * SWI prolog * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> [[Category:Server setup]] 4481476726491a330d17a0e43bfb0717aebcd4fa 1860 1859 2008-12-17T23:04:44Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * VMWare server * Google Earth * VUE * NetLogo * MS core fonts === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> [[Category:Server setup]] 5b57cda018a69af1da4cc375adfe6f9de6dd406d 1868 1860 2008-12-27T17:18:37Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> [[Category:Server setup]] 9eb46f44259a842747f1f1e5425dbfdd424c0dc9 0 1538 1850 1807 2008-11-24T09:42:39Z Neil 2 /* Installation of Rails */ Added link for installing MySQL wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev ruby-extras :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev ruby-extras * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 47796926802d9df1dde002bfe9647882efc64918 0 1545 1851 1784 2008-11-24T09:42:51Z Neil 2 Added installation command wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server The only thing to do here is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords. The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <tt>hostname</tt> is the name of this host, e.g. <tt>server.domain.tld</tt>. * Add passwords to the <tt>root</tt> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); And that's it! == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] 4e0daa517dad9a1a032c3c64fad7f6c2853cc757 1864 1851 2008-12-20T23:42:28Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server The only thing to do here is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <tt>hostname</tt> is the name of this host, e.g. <tt>server.domain.tld</tt>. * Add passwords to the <tt>root</tt> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); And that's it! == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] 549a989f4e96bc035236c18f45a22e05fd4ed872 0 1536 1853 1834 2008-11-26T22:17:34Z Neil 2 /* See also */ Added troubleshooting guide wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Backup setup|Backup]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <tt>/etc/samba/smb.conf</tt> (note that you'll have to skip around the file to find all the settings) [global] ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of workgroup = myworkgroup ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. unix password sync = no #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. * [http://us6.samba.org/samba/docs/Samba24Hc13.pdf Samba troubleshooting guide] (pdf) [[Category:Server setup]] 1bc120e3ff028c7529fbe1aca98ec6f37473557a 1856 1853 2008-11-27T13:50:01Z Neil 2 Added note about logging into machine first wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Backup setup|Backup]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <tt>/etc/samba/smb.conf</tt> (note that you'll have to skip around the file to find all the settings) [global] ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of workgroup = myworkgroup ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. unix password sync = no #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart Note that each user will need to log into the Samba server machine directly (i.e. via the GUI at that machine) to go through the 'first login mandatory password reset' rigmarole. Once that's done, they'll be able to use their Samba share. == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. * [http://us6.samba.org/samba/docs/Samba24Hc13.pdf Samba troubleshooting guide] (pdf) [[Category:Server setup]] e4d4a2513fde10bdc2eb9e4a9162f858ea107e51 0 1515 1861 1759 2008-12-17T23:06:25Z Neil 2 Added router setup page wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * ''[[Ruby on Rails setup]]'' | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[CUPS server setup|Printer server (CUPS)]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] 5ccec5404b7ffc47b216fc3a802547e8ef3b5be9 0 1516 1862 1712 2008-12-17T23:07:52Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Router setup]] |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/products/WhatIsUbuntu/serveredition Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a couple of GB each for <tt>/var</tt> and <tt>/tmp</tt>, and everything else just goes into one large partition. The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <tt>build-essentials</tt> (for basic software development) and <tt>openssh-server</tt> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. [[Category:Server setup]] cdbc83ff541abd37365d5b66f022ad581f68a72d 0 1553 1863 2008-12-17T23:21:18Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| None |align="center" width="50%"| [[OS installation]] |} The router needs a bit of setting up to allow all the server services to be available. This should be done in stages, as things are installed on the server. This document collects them all together for convenience. == Get it working == The first step is to simply get the router working, so that it connects to your ISP and can handle satellite machines connecting via DHCP. == DHCP ranges == In order for the machines with [[Hostname and IP setup|fixed IP numbers]] to avoid stomping all over the router's DHCP server, you need to ensure that some addresses on the local network are not available for allocation by the router's DHCP server. Configure the router so that addresses from 192.168.1.250 to 192.168.1.254 are outside the DHCP pool. == DNS rerouting == By default, the router uses the ISP's DNS servers to find addresses for its DHCP-connected machines. Unfortunately, doing this means that these machines won't be able to find the addresses of the various machines on the LAN. Once the [[DNS server setup|DNS servers]] are set up, change the router's settings so that it no longer gets DNS server information from the ISP, but uses the nameservers you specify: specify the ones you've just set up. == Port forwarding == To make the various services you've set up available to the outside world, you'll need to pass requests from the outside world into the server. The router should have a 'port forwarding' facility or similar, where incoming traffic to the router on a particular port is passed to a specified IP number and port on the LAN. Once you've set up a service, you can open the port for it. The ports you'll need to open are: * Port 25 for [[Postfix server setup|SMTP mail transport]] * Port 80 for [[Web server setup|serving Web pages]] * Port 443 for secure web pages, such as used by [[Webmail setup|webmail]] You may want to consider opening port 22 (SSH) which will allow secure remote logins to a machine. [[Category:Server setup]] 65ad2aff79417e6119b10e3643a80cc57182f9cc 0 1544 1865 1783 2008-12-21T00:32:24Z Neil 2 Moved references to a 'see also' section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers forwarders { 1.2.3.4; 1.2.3.5; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld NS desktop.domain.tld ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld : (and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/bind/run root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run * Modify the file <tt>/etc/default/bind9</tt> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * Modify <tt>/etc/default/syslogd</tt> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * Make AppArmor allow BIND access to the chroot jail. Modify <tt>/etc/apparmor.d/usr.sbin.named</tt> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/run/bind/run/named.pid w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, Sysklogd, and BIND: root@server:~# /etc/init.d/apparmor restart root@server:~# /etc/init.d/sysklogd restart root@server:~# /etc/init.d/bind9 restart * Check <tt>/var/log/syslog</tt> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] e5df5e7601c83f638039ed522f05ae91bbf228be 1866 1865 2008-12-21T00:34:32Z Neil 2 /* Getting the servers running */ Updated named.conf.options contents wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld NS desktop.domain.tld ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld : (and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/bind/run root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run * Modify the file <tt>/etc/default/bind9</tt> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * Modify <tt>/etc/default/syslogd</tt> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * Make AppArmor allow BIND access to the chroot jail. Modify <tt>/etc/apparmor.d/usr.sbin.named</tt> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/run/bind/run/named.pid w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, Sysklogd, and BIND: root@server:~# /etc/init.d/apparmor restart root@server:~# /etc/init.d/sysklogd restart root@server:~# /etc/init.d/bind9 restart * Check <tt>/var/log/syslog</tt> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 87a5ef00821f93712ffdd8f3a4832eb5ddd8108d 0 1540 1867 1795 2008-12-22T08:04:57Z Neil 2 /* Anacron timing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <tt>/etc/cron.daily/makedailybackups</tt>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +1 -exec rm -f {} \; and <tt>/etc/cron.monthly/makemonthlybackups</tt> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -ctime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <tt>--absolute-names</tt> would restore the files to paths anchored at root. == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <tt>/etc/cron.daily/makedailybackups</tt>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +2 -exec rm -f {} \; # Create file backups tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, adjust the datestamps in the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. [[Category:Server setup]] 92298707994beab3ab60f7500df02becfd1c5d4b 0 1542 1869 1781 2009-01-05T16:41:53Z Neil 2 Added HPLIP update instructions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <tt>localhost</tt>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <tt>/etc/cups/cupsd.conf</tt> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock and add the <tt>Allow @LOCAL</tt> lines in the two <tt>Location</tt> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# /etc/init.d/cupsys restart You should then be able to point a webbrowser at <tt>desktop.server.tld:631</tt> or <tt>server.domain.tld:631</tt> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <tt>Administration</tt> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is only supported in HPLIP 2.8.10 and higher, but Ubuntu 8.04 only has HPLIP 2.8.2. So I need to update it. NB: Update the software before attaching the printer. * Remove the existing HPLIP installation. root@desktop:~# apt-get purge hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions] That's it! == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <tt><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></tt>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. [[Category:Server setup]] 659ba88b8f67f27323b61e281ab5a9f6b1ce5025 1870 1869 2009-01-06T09:20:27Z Neil 2 /* Windows use of printers */ Added some instructions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <tt>localhost</tt>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <tt>/etc/cups/cupsd.conf</tt> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock and add the <tt>Allow @LOCAL</tt> lines in the two <tt>Location</tt> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# /etc/init.d/cupsys restart You should then be able to point a webbrowser at <tt>desktop.server.tld:631</tt> or <tt>server.domain.tld:631</tt> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <tt>Administration</tt> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is only supported in HPLIP 2.8.10 and higher, but Ubuntu 8.04 only has HPLIP 2.8.2. So I need to update it. NB: Update the software before attaching the printer. * Remove the existing HPLIP installation. root@desktop:~# apt-get purge hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions] That's it! == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <tt><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></tt>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code>http://printers.domain.tld:631/printers</code>. Select the printer you want, such as <code>http://printers.domain.tld:631/printers/printer-raw</code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] 9e4b43dbdcad0fb946bf4655d49f7691ac5137bd 0 1554 1871 2009-01-08T20:46:08Z Neil 2 New page: <blockquote>A [[:Category:HeroQuest|HeroQuest]] Episode by Neil Smith</blockquote> == Introduction == This is an episode for [[:Category:HeroQuest|HeroQuest]]. It was written to be playe... wikitext text/x-wiki <blockquote>A [[:Category:HeroQuest|HeroQuest]] Episode by Neil Smith</blockquote> == Introduction == This is an episode for [[:Category:HeroQuest|HeroQuest]]. It was written to be played by five or six players (four at a pinch) in a single three to four hour session, typical of convention games. Six [[Peace Process characters|pre-generated characters]] are provided. It is designed as a drama, rather than a thriller: the engine of the scenario lies in the characters, their motivations, and the conflicts between them. The focus should be not so much on the events that unfold during the scenario as on how the heroes react and interact as a result of those events. I advise you to understand the inter-hero tensions fully before you run the scenario. Also, when running this episode, ensure that you give plenty of time for the players to chew the scenery and explore their heroes. This episode is set in an anonymous valley in the Quivin Mountains in Sartar. All the heroes are typical Orlanthi from two neighbouring clans as they attempt to bring an end to the feud that has damaged them for decades. This episode requires the '''HeroQuest''' core rulebook. '''Thunder Rebels''' and '''Storm Tribe''' are both extremely useful for background information. The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll probably need a copy of this myth as a handout for play. I'm told that '''Barbarian Adventures''' may come in handy, but I don't actually own it! {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Many contests are described in a short form, as '''''task''' (resistance): typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. |} == Backstory == {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Read this to the players at the start of the game. Don't mention that Sarotar and Gringle are conspiring against the other heroes. |} Until twenty years ago, the neighbouring Singing Rock and Hiord clans got on tolerably well; well enough for the young Singing Rock woman, '''Estave''', to marry '''Sarotar''', her true love, and leave her birth clan to join him in the Hiord clan. Soon after that, relations between the two clans began to break down and the usual cycle of cattle raids became more vicious on both sides. Eventually, a feud was declared and the fyrds skirmished. In that first battle, Brol the farmer was killed. '''Dorasa''', his wife, managed to have him resurrected but he had the Relife sickness. Brol turned to Humakt and became '''Mortality''', Champion of the Singing Rock clan. Since then, the feud has rumbled on, sometimes as a "cold war," sometimes erupting into violence. Recently, Estave and Dorasa have managed to broker a deal that will lead the two clans to peace. The two clans have agreed to end the feud and have charged Iristha (an Issarian from the Singing Rock clan) and Mikhil (a Lhankor Mhyite from the Hiord clan) to re-enact the "Issaries the Conciliator" myth. The quest will start in the Hirod tula tomorrow morning and finish atop the Singing Rocks tomorrow evening. The purpose of the quest is to discover the terms that will end the feud. When the heroquesters return, they will bring the resolution with them; both clans have promised to honour the terms on their return. The restoration of normal relations will be cemented by the marriage of '''Askos''' and Levru, a marriage arranged by Estave. Both clans have committed all their magical resources to this quest and, if either the quest or the marriage fail to happen tomorrow, the damage to both clans will be dreadful. The heroes have been charged with ensuring that the marriage takes place tomorrow evening, atop the Singing Rocks, when the heroquesters arrive. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |But all is not well. Sarotar and '''Gringle''', warleaders of the two opposing clans, have conspired to derail the quest by kidnapping Levru and ambushing the heroquesters on the Hero Plane. They each have their own reasons for this, but both agree that war is better than peace. |} == Summary == When the episode starts, all the heroes are at the edge of the Singing Rock tula, waiting for Levru to arrive. They are to form the honour guard for Levru as she travels to her new home there. Just as she arrives, her party is ambushed by a band of Gagarthi bandits (arranged by Sarotar), who kidnap Levru. The heroes will probably track the bandits, and may or may not discover their camp. They may be able to interrogate at least one Gagarthi and learn that they were hired to perform this ambush by someone in one of the clans. The heroes may try other ways of discovering more about the Gagarthi, but to little avail. The next day, during the Windsday rituals at the Singing Rocks, one of the clan's new weaponthanes is driven wild by impests and yavorlings and falls to his death from the Rocks. The heroes should soon discover that other weaponthanes are implicated in the plot to kidnap Levru. The heroes race to the weaponthanes' training ground where they discover Levru (unharmed) and a portal to the Hero Plane. The heroes step through the portal and foil an attempt by the conspirators to ambush and kill the heroquesters Iristha and Mikhil. At some point in all this, Gringle and Sarotar reveal themselves as the plot's ringleaders, probably as they attack the other heroes. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The players are almost certainly going to move a long way from the events described here. Let them. However, the characters carry the conflict with them (the important one is between Gringle & Sarotar and the rest of the heroes), and there is a tight deadline to work to, so they should bring themselves back on track. So long as you keep that in mind, you should be able to improvise interesting events around the heroes, whatever the players have them do. As examples, in one playtest, Sarotar confessed all just after the death at the Singing Rock, shopped Gringle (an NPC in that game) and asked for a chance to put right his mistakes: a very Orlanthi thing to do. In another session, the climax was when Mortality came to fulfil his vow to kill the defeated Sarotar, only for Estave to throw herself over her husband's body: Mortality killed them both. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Don't worry too much about the pacing in this episode. The heroes are likely to move through most of it quite quickly, but you're unlikely to finish too soon. The final scene, where Gringle , Sarotar, and the other conspirators stab the other heroes in the back, will take quite a while to play out. |} == The Episode == === Act I: Introduction === ==== Scene I: The Gagarthi Raid ==== ''Key Points: The Heroes get to strut their stuff, and some Gagarthi kidnap Levru.'' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |This scene is more about giving the players a chance to give their new characters a work out than influencing the plot. |} The scene opens with the heroes standing on the edge of the Singing Rock tula, waiting for Levru to arrive. They have all just left the Singing Rock stead where they have been preparing for Levru's arrival. Mortality has witnessed the Orlanthi Greeting that gave Sarotar the Blanket gift and Estave the Meat gift; this means that the Singing Rock clan has pledged hospitality and safety to both of them ('''Thunder Rebels''', pp. 34-36). The heroes are to be Levru's honour guard as she crosses the Singing Rock tula. She is being brought across the Hiord tula by an honour guard of Hiord weaponthanes. Just as the Hiord party comes into earshot, the heroes can use their (magical) perception abilities to '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. If they succeed, describe the feel of a hot, dry wind, the sand stinging their cheek, and the smell of hot flint. As soon as they think they have been rumbled, or when the Hiord party turn up, the Gagarthi attack. Many of them attack the Hiord party; the remainder try to hold back the heroes. The heroes face one Gagarthi leader per hero and two henchmen per hero , together with the odd whirlvish or two. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Gagarthi Leader: Sword & Shield Fighting 14W, (+3 auto-augment from Tough Outlaw = 5W), Walk on Winds 6W, Stinging Winds 2W, Tough Outlaw 5W, 39AP + 27AP + 27AP (from two henchmen) = 93AP. |- |Gagarthi Henchman: Sword & Shield Fighting 5W, Stinging Winds 18, Tough Outlaw 19; treat as followers, providing 27AP each. |- |Whirlvish: Abrade 10W, Spirit combat 15, 32AP. |} If the heroes win this encounter, they drive their assailants back into the Hiord party, who then capture and escape with Levru. If the heroes lose, they are forced back and fail to prevent the Gagarthi from kidnapping Levru. Try to ensure that none of the Gagarthi leaders are captured. ==== Scene II (Plot Turn I): Kidnap and Recriminations ==== ''Key Points: The heroes discover that Levru has been taken, and that the peace deal is not a foregone conclusion.'' When the Gagarthi leave, the heroes will want to investigate the Hiord party. They will discover that the 12 Hiordlings have been defeated, many of them are injured, and they have just noticed that Levru has been taken, snatched by a wind-walking Gagarthi. Many of the Hiordlings will blame the heroes for the attack, claiming that they were on Singing Rock lands and that the Singing Rocks are responsible for the injuries. Someone might even suggest that they wanted the Gagarthi to attack (though they have no evidence to support this: it's just hot-headed talking). Tempers can be calmed in a number of ways: tending to the wounded, boasting of how the Gagarthi were defeated, vows to avenge the terrible crime. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |As the heroes take stock of the situation, reinforce that the marriage between Levru and Askos is an essential part of the peace deal. If the marriage doesn't occur at the prescribed place and time, the feud will continue at great cost to both clans. |} === Act II: Conflict === ==== Scene I: Hunting the Wild Hunters ==== ''Key Points: The heroes discover that the Gagarthi had an informer from one of the two clans.'' The ball is now in the heroes' court. They are likely to want to gain some information about the Gagarthi and Levru: where they are, why they attacked, how they knew, and what are they planning for Levru. There are a few avenues that could be open to them. There could be Gagarthi prisoners to interrogate. Some Gagarthi fled on foot and can be tracked. The god-talker heroes may want to use divinations. Also, someone will have to tell the Singing Rock clan the bad news of Levru's kidnap. * ''Interrogating Gagarthi'': The bandits are fanatics, and all declared outlaw. They know that if they are ever brought to justice they are almost certain to be executed, so they have no real reason to co-operate. This makes interrogation difficult. The heroes know this (inform the players if need by), but they are likely anyway to want to interrogate them anyway. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate Gagarthi Bandits''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Resistance'': 12W2 |- |''Complete or Major Defeat'': The bandit either commits suicide or manages to snatch a hostage. |- |''Other Defeat or Tie'': The bandit reveals nothing. |- |''Marginal Victory'': The bandit hints that they knew that someone would be here. |- |''Minor Victory'': The bandits were waiting specifically for this group of Hiord clansfolk. |- |''Major Victory'': The Gagarthi were hired to kidnap “the girl.” |- |''Complete Victory'': This was definitely an inside job, connected to some big ritual. He doesn't know the identity of the insider. |} Heroes may '''''Recognise the Bandit''' (15W): Relationship to Clan'' and recall the bandit's name and deeds, and reduce the bandit's resistance according to the result of that contest. * ''Tracking the Gagarthi'': The wind-walking Gagarthi leaders are the ones who stole Levru, and they are difficult to track. However, the other Gagarthi who escaped are easier to follow: '''''Track the Gagarthi''' (3W): Track, Scout (-5), Singing Rock Geography (-10), Hiord Geography (-10)''. Any victory allows the heroes to follow the trail. It soon gets easier to follow once the wind-walking bandits come to earth. After a few hours of hard walking over some increasingly rough terrain (up and down some some very steep and heavily wooded slopes), the heroes come to the Gagarthi's (deserted) camp. The camp has been packed up recently and the firepit it still hot. Observant heroes will notice two fresh trails leaving the camp, one more heavily used than the other. Both peter out after a mile or so as they pass over bare, rocky ground in the fading light. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Their job done, the Gagarthi have been paid off and have left the area. Levru has been taken by some weaponthanes to the training camp in the Singing Rock tula. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |One potential problem point is persuading the heroes that they've lost the Gagarthi's trail. Stress the difficulty of following the trail, the worsening conditions, and time pressure the heroes are under to force them into trying another tack. |} * ''Asking the gods'': The heroes may ask the gods, or at least the Singing Rock clan wyter, where Levru and the Gagarthi are. (The Singing Rocks are the best bet, as most of the heroes have magical connections there they can call on.) Let the heroes go to town in describing the preparations, and involve the heroes in persuading the clan to support them and provide community support and objects to sacrifice. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Ask the Gods about Levru and the Gagarthi''' |- |''Appropriate abilities'': Pray to Orlanth, Pray to Ernalda, Devotee of <Deity> (-5) |- |''Modifiers and Augments'': Length of ritual (+3/hr), <Deity> Myths, various Relationships and Personality traits (for emotional involvement), various Relationships (for community support), auspicious location (e.g. the Singing Rocks) (+10), value of sacrifices (up to +10), not a member of the Singing Rock clan (-20), penalties for fatigue (contest based on time and exertion up to now). See also the Ritual Magic section, HeroQuest pp. 101-2. |- |''Resistance'': 10W3 |- |''Complete, Major, or Minor Defeat'': The heroes fail to learn anything. |- |''Marginal Defeat or Tie'': The Singing Rock wyter tells of a disturbance in the clan. |- |''Marginal Victory'': There is something wrong in the clan. Neither Levru nor the Gagarthi are touched by the wind or the earth within the Singing Rock tula. |- |''Minor Victory'': There is something wrong in the clan. Levru is within the tula; the Gagarthi are not. |- |''Major Victory'': There is discord in the Song of the Rocks, discord in the very heart of the tula. Levru is hidden within the tula. |- |''Complete Victory'': Kinstrife! Kinstrife! Kinstrife brought on by people both here and elsewhere! A vision of Levru being bound by a man with Singing Rock clan tatoos. |} By the time the rituals are complete, the heroes are liable to be too fatigued to do much more. They should go to bed disheartened that the peace deal is in danger of failure. ==== Scene II (Midpoint): When Impests Attack! ==== ''Key Points: The conspiracy is revealed when a weaponthane is driven to his death by Impests.'' At dawn the next morning, the heroes join the assembling clan atop the Singing Rocks. The mood is sombre as the clan contemplate the impending failure of the heroquest. The heroes may contemplate trying to persuade Sarostip that the quest should be abandoned, but they know that the consequences abandoning it would be worse than the consequences of failure, assuming that they could get word to Iristha nd Mikhil in time. The heroes may '''''notice that some weaponthanes are missing''' (2W): Relationship to warband, Relationship to clan (-8)'', though they won't necessarily know what that means. As the early morning worship starts, there is a scream from in the congregation as one of the weaponthanes starts to flail around wildly. Characters may '''''sense the clustering agents of reprisal''' (14): Sense gods nearby, Piety (-5)'' as the man is attacked by yavorlings and harulings (sparkers and impests) ('''Thunder Rebels''', pp. 81-82). As he attempts to flee, he runs blindly off the edge of the cliff and falls to smash on the rocks far below. Everyone is shocked, both because of the sudden and violent death, and for the calamitous omen it represents. The agents of reprisal follow him down and circle his corpse. They then rise again to swirl around the huddled clan. Perhaps their work is not yet done... ==== Scene III: The Conspiracy Uncovered ==== ''Key Points: The heroes discover that the conspiracy is led by the weaponthanes.'' All the Singing Rock heroes will recognise the dead man as Willem, a new weaponthane. They are likely to want to have pointed words with his colleagues, particularly in light of their suspicions that there is a traitor in the clan. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate the Weaponthanes''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Modifiers and Augments'': Threats of violence (-20 to +20, depending on contest v. resistance of 15W), promises of amnesty (as threats of violence), Relationship to warriors. |- |''Resistance'': 5W2 |- |''Complete or Major Defeat'': The militia manage to place the blame on the few Hiord clansmen present and incite a lynch mob. |- |''Other Defeat or Tie'': The weaponthanes reveal nothing. |- |''Marginal Victory'': Someone says that the weaponthanes have been plotting something, but there are no more details. |- |''Minor Victory'': A warrior with a mouthful of dirt blurts out that “They forced me to keep quiet!” and reveals that some weaponthanes (inlcuding Gringle, if he's an NPC) want the feud to continue. |- |''Major Victory'': The weaponthanes reveal that some of them have decided the feud should continue, and are conspiring to ensure that it does. |- |''Complete Victory'': The weaponthanes tell everything, including the involvement of Sarotar and Gringle, then attack them! |} ==== Scene IV (Plot Turn II): To the Training Ground ==== ''Key Points: The heroes rescue Levru and enter the Hero Plane.'' The heroes will either have evidence that the weaponthanes are involved, or suspect that this is the case. In either case, they are likely to want to pay a visit to their training ground, a mile or so from the stead. The training ground is a small clearing, about a hundred yards across, on a level piece of ground in the hills above the Singing Rocks. Around the field are archery butts, straw-stuffed dummies for sword practice, and other such paraphernalia. When the heroes arrive, the field is deserted. One one side of the field is the weaponthanes' longhouse. It is silent and dark, though smoke is rising from the smoke hole. The only doors are in the middle of the long side facing the field; they are closed and barred from the inside. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Play up the paranoia while the heroes are moving into the longhouse. They shouldn't know the place is deserted until they rush the longhouse. |} When the heroes enter the longhouse, three things are immediately apparent. First is the bullock in the middle of the floor, lying in a pool if its own blood. Second is Levru, gagged and tied to a chair at one end of the longhouse. Third is a large portal to the Hero Plane at the other end of the longhouse. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The portal looks rather like the Stargate from the TV series, if that would help your players visualise it. |} The bullock has been ritually slaughtered as part of a ceremony. Any god-talker will be able to deduce that the ritual was probably the one to open the portal. Levru is unharmed and will tell the heroes that, about an hour ago, a group of four weaponthanes opened the portal. They were talking about ambushing Iristha and Mikhil on the Hero Plane. Levru will tell the heroes that she recognised two of the weaponthanes as being Hiord clansmen; the remainder had Singing Rock clan tattoos. The heroes should realise that they are the only ones who can save the peace treaty, by eliminating the weaponthanes so that Iristha and Mikhil can complete their hero quest. The heroes will have to enter the hero plane. === Act III: Resolution === ==== Scene I: Rumble on the Hero Plane ==== ''Key Points: The heroes discover the full extent of the conspiracy, and it is either defeated or it isn't.'' When the heroes enter the hero plane, they can easily track the weaponthanes through a small patch of forest to where they are lying hidden above a trail, waiting to ambush Iristha and Mikhil when they pass through into the Big Teeths' lands. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Emphasise the hyper-realistic nature of the hero plane. Colours are more vibrant; vision is more acute; smells are more intense; sounds are more moving. |} The conspirators, Baranwulf, Illig, Kagrad, and Verlain (Ranged attack 16W, Close attack 6W2, Resist Magic 14W, Hardened Consiprator 15W, all after augments) will attack the heroes if attacked themselves, or if they are being beaten in a battle of wits and words. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The hero plane is the world of myth, and you should bring this out in your desciptions. When the conspirators attack, they are taking the part of the Long Teeth; their leather armour regrows stiff, spotted fur; it's difficult to hear words in their animalistic snarling; in the confusion of melee, are they attacking with their swords or their foot-long canines? Heroes, particularly Dorasa and Estave, trumpet and bellow and crush their enemies beneath their feet. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The normal physical rules don't entirely apply on the hero plane. If a character explicitly invokes a myth to justify their actions, give them a bonus of +5 or so. If they continue to invoke the same myth successfully, increase the bonus or allow them to decrease their opponents' bonus. The same bonus applies to everyone else who invokes the same myth. Some mythlets are in the appendix; the conspirators are invoking the “Big Teeth kill the Intruders” myth. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Just as Earth defeats Storm in the elemental progression, so Earth cultists have an advantage over Storm cultists in the hero plane. Give Dorasa and Estave a bonus of +5 to resist all actions by an Orlanth cultist, rising to +10 when resisting magic with magic (alternatively, use the rules from Masters of Luck and Death, p. 52) |} At some point in this confrontation, if not before, the players of Gringle and Sarotar will want to reveal their true objectives and stab the other heroes in the back. Let them. The whole episode has been building up to this struggle between the player heroes, and its is their actions that will determine whether the peace process succeeds or not. ==== Scene II: Denouement ==== If the conspirators are victorious, they move on to ambush and kill Iristha and Mikhil. Normal life in the two clans is shattered as open, total war breaks out between them. The weaponthanes gain power and prestige, but at a dreadful cost. If the heroes are successful, they hear the sounds of Issaries and Lhankor Mhy passing below just as the conspirators are defeated. The heroes can return to the Singing Rocks in time for the wedding between Askos and Levru. If the heroes succeed, but Askos cannot or will not marry Levru, another bachelor can be found but the resulting peace treaty will be neither as strong nor as long-lasting. == Acknowledgements == I'd like to thank Rob Davies for his feedback from running and playtesting this scenario. I'd also like to thank Tim Ellis, Newt Newport, and Paul Kay for their insightful reviews and comments. == Appendix: Mythlets == The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll need a copy of this myth as a handout for play. === Orlanth Banishes The Thieves === One day, Orlanth came home from a day's work in the fields to find that his favourite neck torc was missing. He looked all over his stead for it: he lifted up the mattresses, he flew up into the rafters, he swirled into all the corners of the longhouse, but he couldn't find his neck torc anywhere. Later, at dinner, he heard Vadrus, his brother, boasting to his friends in the corner of the longhouse. Orlanth left his dinner to see what Vadrus was going on about this time. When he got there, he saw that Vadrus was wearing Orlanth's favourite neck torc! "That’s mine!" shouted Orlanth. "Give it back." "No," said Vadrus. "I ... found it, in the field. It’s mine now." "But my laws say that even when something is lost, it still belongs to its owner. It’s mine." "I claim it by right of conquest!" bellowed Vadrus. "If you want it, come an take it!" But instead of reaching for his sword, Orlanth reached for his lawstaff. "This is my law," he said, "and you will obey it!" And with that he cracked Vadrus's head with the staff and Vadrus dropped the torc. Vadrus and his friends ran out of the longhouse. Orlanth chased them all the way to the edge of his tula and told them never to come back. === Humakt Slays the Enemies === After Humakt had reclaimed Death, and after Orlanth had given him a place in his hall, Humakt faced many enemies. Whenever the Sky Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Night Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Plant Tribe attacked, Humakt killed their warriors and drove them back. But when others in the Storm Tribe attacked, no-one would fight them. Even thought the Storm Tribe were enemies, they were still kin and Orlanth would not attack them. Vadrus, and Valind, and even Ragnaglar came to Orlanth's tula to fight and steal, but Orlanth would not risk kinstrife by driving them off. Humakt said that he had no kin, and so could attack the enemies, but Orlanth always told him to stay in the longhouse. Humakt worried about this. He had given his word to Orlanth always to obey his commands, but he had also given his word to Orlanth always to defend his tula. Now his two oaths were in conflict. Humakt thought long and hard about this, and then decided what to do. Soon, the winds started to blow hot and cold and fierce. It was an omen that the bad members of the Storm Tribe were coming to attack again. As soon as he felt the wind, Humakt left Orlanth’s hall. That way, when Orlanth gave the orders not to attack Valind, Humakt wasn’t there to hear them. Humakt was free to carry out his oath to defend Orlanth’s tula and, when Valind stepped onto the tula, Humakt stepped up to him and killed him. === Ernalda Heals the Tribe === Even after Ernalda told the Orlanth to make the Storm Tribe, there were problems. Even after Ernalda told Orlanth what laws to make, there were problems. Even after Ernalda told Orlanth how to judge the lawsuits, there were problems. Ernalda sat and thought about the problems and what she could do about them. She discovered that the problems were always worse just after Issaries had visited, so the next time he came, she watched very carefully. She saw that Issaries had many beautiful and wonderful things he had found in foreign lands. She saw that everyone in the Tribe wanted all the things, but Issaries wouldn’t just give them away. No, Issaries traded for them, so that everyone had to give something up to get something new. This made a lot of people unhappy, because they weren't any better off after Issaries had been than they were before, and now they were angry that there were lots of things that Issaries had that they didn’t, and that some people had more things than others. Ernalda knew what to do. One night, when everyone was sleeping, she took away all the things that Issaries had brought and hid them in her secret place. The next morning when everyone woke up, they couldn't find the things Issaries had brought and they were all very angry. They all came to Orlanth’s hall and demanded that he did something about it. Then Ernalda spoke to them and showed them that they still had the most important thing of all, and that was the Storm Tribe. Everyone realised that Ernalda was speaking the truth, and they were all happy again. Then Ernalda took everything out of her secret place, and everyone agreed to share them among themselves. == Characters == [[Peace Process characters|Characters for this episode]] are on a separate page. If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] 0466a3bf135e03c8dd228e3255729eb27c072242 1874 1871 2009-01-08T21:17:34Z Neil 2 /* Characters */ wikitext text/x-wiki <blockquote>A [[:Category:HeroQuest|HeroQuest]] Episode by Neil Smith</blockquote> == Introduction == This is an episode for [[:Category:HeroQuest|HeroQuest]]. It was written to be played by five or six players (four at a pinch) in a single three to four hour session, typical of convention games. Six [[Peace Process characters|pre-generated characters]] are provided. It is designed as a drama, rather than a thriller: the engine of the scenario lies in the characters, their motivations, and the conflicts between them. The focus should be not so much on the events that unfold during the scenario as on how the heroes react and interact as a result of those events. I advise you to understand the inter-hero tensions fully before you run the scenario. Also, when running this episode, ensure that you give plenty of time for the players to chew the scenery and explore their heroes. This episode is set in an anonymous valley in the Quivin Mountains in Sartar. All the heroes are typical Orlanthi from two neighbouring clans as they attempt to bring an end to the feud that has damaged them for decades. This episode requires the '''HeroQuest''' core rulebook. '''Thunder Rebels''' and '''Storm Tribe''' are both extremely useful for background information. The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll probably need a copy of this myth as a handout for play. I'm told that '''Barbarian Adventures''' may come in handy, but I don't actually own it! {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Many contests are described in a short form, as '''''task''' (resistance): typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. |} == Backstory == {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Read this to the players at the start of the game. Don't mention that Sarotar and Gringle are conspiring against the other heroes. |} Until twenty years ago, the neighbouring Singing Rock and Hiord clans got on tolerably well; well enough for the young Singing Rock woman, '''Estave''', to marry '''Sarotar''', her true love, and leave her birth clan to join him in the Hiord clan. Soon after that, relations between the two clans began to break down and the usual cycle of cattle raids became more vicious on both sides. Eventually, a feud was declared and the fyrds skirmished. In that first battle, Brol the farmer was killed. '''Dorasa''', his wife, managed to have him resurrected but he had the Relife sickness. Brol turned to Humakt and became '''Mortality''', Champion of the Singing Rock clan. Since then, the feud has rumbled on, sometimes as a "cold war," sometimes erupting into violence. Recently, Estave and Dorasa have managed to broker a deal that will lead the two clans to peace. The two clans have agreed to end the feud and have charged Iristha (an Issarian from the Singing Rock clan) and Mikhil (a Lhankor Mhyite from the Hiord clan) to re-enact the "Issaries the Conciliator" myth. The quest will start in the Hirod tula tomorrow morning and finish atop the Singing Rocks tomorrow evening. The purpose of the quest is to discover the terms that will end the feud. When the heroquesters return, they will bring the resolution with them; both clans have promised to honour the terms on their return. The restoration of normal relations will be cemented by the marriage of '''Askos''' and Levru, a marriage arranged by Estave. Both clans have committed all their magical resources to this quest and, if either the quest or the marriage fail to happen tomorrow, the damage to both clans will be dreadful. The heroes have been charged with ensuring that the marriage takes place tomorrow evening, atop the Singing Rocks, when the heroquesters arrive. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |But all is not well. Sarotar and '''Gringle''', warleaders of the two opposing clans, have conspired to derail the quest by kidnapping Levru and ambushing the heroquesters on the Hero Plane. They each have their own reasons for this, but both agree that war is better than peace. |} == Summary == When the episode starts, all the heroes are at the edge of the Singing Rock tula, waiting for Levru to arrive. They are to form the honour guard for Levru as she travels to her new home there. Just as she arrives, her party is ambushed by a band of Gagarthi bandits (arranged by Sarotar), who kidnap Levru. The heroes will probably track the bandits, and may or may not discover their camp. They may be able to interrogate at least one Gagarthi and learn that they were hired to perform this ambush by someone in one of the clans. The heroes may try other ways of discovering more about the Gagarthi, but to little avail. The next day, during the Windsday rituals at the Singing Rocks, one of the clan's new weaponthanes is driven wild by impests and yavorlings and falls to his death from the Rocks. The heroes should soon discover that other weaponthanes are implicated in the plot to kidnap Levru. The heroes race to the weaponthanes' training ground where they discover Levru (unharmed) and a portal to the Hero Plane. The heroes step through the portal and foil an attempt by the conspirators to ambush and kill the heroquesters Iristha and Mikhil. At some point in all this, Gringle and Sarotar reveal themselves as the plot's ringleaders, probably as they attack the other heroes. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The players are almost certainly going to move a long way from the events described here. Let them. However, the characters carry the conflict with them (the important one is between Gringle & Sarotar and the rest of the heroes), and there is a tight deadline to work to, so they should bring themselves back on track. So long as you keep that in mind, you should be able to improvise interesting events around the heroes, whatever the players have them do. As examples, in one playtest, Sarotar confessed all just after the death at the Singing Rock, shopped Gringle (an NPC in that game) and asked for a chance to put right his mistakes: a very Orlanthi thing to do. In another session, the climax was when Mortality came to fulfil his vow to kill the defeated Sarotar, only for Estave to throw herself over her husband's body: Mortality killed them both. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Don't worry too much about the pacing in this episode. The heroes are likely to move through most of it quite quickly, but you're unlikely to finish too soon. The final scene, where Gringle , Sarotar, and the other conspirators stab the other heroes in the back, will take quite a while to play out. |} == The Episode == === Act I: Introduction === ==== Scene I: The Gagarthi Raid ==== ''Key Points: The Heroes get to strut their stuff, and some Gagarthi kidnap Levru.'' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |This scene is more about giving the players a chance to give their new characters a work out than influencing the plot. |} The scene opens with the heroes standing on the edge of the Singing Rock tula, waiting for Levru to arrive. They have all just left the Singing Rock stead where they have been preparing for Levru's arrival. Mortality has witnessed the Orlanthi Greeting that gave Sarotar the Blanket gift and Estave the Meat gift; this means that the Singing Rock clan has pledged hospitality and safety to both of them ('''Thunder Rebels''', pp. 34-36). The heroes are to be Levru's honour guard as she crosses the Singing Rock tula. She is being brought across the Hiord tula by an honour guard of Hiord weaponthanes. Just as the Hiord party comes into earshot, the heroes can use their (magical) perception abilities to '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. If they succeed, describe the feel of a hot, dry wind, the sand stinging their cheek, and the smell of hot flint. As soon as they think they have been rumbled, or when the Hiord party turn up, the Gagarthi attack. Many of them attack the Hiord party; the remainder try to hold back the heroes. The heroes face one Gagarthi leader per hero and two henchmen per hero , together with the odd whirlvish or two. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Gagarthi Leader: Sword & Shield Fighting 14W, (+3 auto-augment from Tough Outlaw = 5W), Walk on Winds 6W, Stinging Winds 2W, Tough Outlaw 5W, 39AP + 27AP + 27AP (from two henchmen) = 93AP. |- |Gagarthi Henchman: Sword & Shield Fighting 5W, Stinging Winds 18, Tough Outlaw 19; treat as followers, providing 27AP each. |- |Whirlvish: Abrade 10W, Spirit combat 15, 32AP. |} If the heroes win this encounter, they drive their assailants back into the Hiord party, who then capture and escape with Levru. If the heroes lose, they are forced back and fail to prevent the Gagarthi from kidnapping Levru. Try to ensure that none of the Gagarthi leaders are captured. ==== Scene II (Plot Turn I): Kidnap and Recriminations ==== ''Key Points: The heroes discover that Levru has been taken, and that the peace deal is not a foregone conclusion.'' When the Gagarthi leave, the heroes will want to investigate the Hiord party. They will discover that the 12 Hiordlings have been defeated, many of them are injured, and they have just noticed that Levru has been taken, snatched by a wind-walking Gagarthi. Many of the Hiordlings will blame the heroes for the attack, claiming that they were on Singing Rock lands and that the Singing Rocks are responsible for the injuries. Someone might even suggest that they wanted the Gagarthi to attack (though they have no evidence to support this: it's just hot-headed talking). Tempers can be calmed in a number of ways: tending to the wounded, boasting of how the Gagarthi were defeated, vows to avenge the terrible crime. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |As the heroes take stock of the situation, reinforce that the marriage between Levru and Askos is an essential part of the peace deal. If the marriage doesn't occur at the prescribed place and time, the feud will continue at great cost to both clans. |} === Act II: Conflict === ==== Scene I: Hunting the Wild Hunters ==== ''Key Points: The heroes discover that the Gagarthi had an informer from one of the two clans.'' The ball is now in the heroes' court. They are likely to want to gain some information about the Gagarthi and Levru: where they are, why they attacked, how they knew, and what are they planning for Levru. There are a few avenues that could be open to them. There could be Gagarthi prisoners to interrogate. Some Gagarthi fled on foot and can be tracked. The god-talker heroes may want to use divinations. Also, someone will have to tell the Singing Rock clan the bad news of Levru's kidnap. * ''Interrogating Gagarthi'': The bandits are fanatics, and all declared outlaw. They know that if they are ever brought to justice they are almost certain to be executed, so they have no real reason to co-operate. This makes interrogation difficult. The heroes know this (inform the players if need by), but they are likely anyway to want to interrogate them anyway. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate Gagarthi Bandits''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Resistance'': 12W2 |- |''Complete or Major Defeat'': The bandit either commits suicide or manages to snatch a hostage. |- |''Other Defeat or Tie'': The bandit reveals nothing. |- |''Marginal Victory'': The bandit hints that they knew that someone would be here. |- |''Minor Victory'': The bandits were waiting specifically for this group of Hiord clansfolk. |- |''Major Victory'': The Gagarthi were hired to kidnap “the girl.” |- |''Complete Victory'': This was definitely an inside job, connected to some big ritual. He doesn't know the identity of the insider. |} Heroes may '''''Recognise the Bandit''' (15W): Relationship to Clan'' and recall the bandit's name and deeds, and reduce the bandit's resistance according to the result of that contest. * ''Tracking the Gagarthi'': The wind-walking Gagarthi leaders are the ones who stole Levru, and they are difficult to track. However, the other Gagarthi who escaped are easier to follow: '''''Track the Gagarthi''' (3W): Track, Scout (-5), Singing Rock Geography (-10), Hiord Geography (-10)''. Any victory allows the heroes to follow the trail. It soon gets easier to follow once the wind-walking bandits come to earth. After a few hours of hard walking over some increasingly rough terrain (up and down some some very steep and heavily wooded slopes), the heroes come to the Gagarthi's (deserted) camp. The camp has been packed up recently and the firepit it still hot. Observant heroes will notice two fresh trails leaving the camp, one more heavily used than the other. Both peter out after a mile or so as they pass over bare, rocky ground in the fading light. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Their job done, the Gagarthi have been paid off and have left the area. Levru has been taken by some weaponthanes to the training camp in the Singing Rock tula. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |One potential problem point is persuading the heroes that they've lost the Gagarthi's trail. Stress the difficulty of following the trail, the worsening conditions, and time pressure the heroes are under to force them into trying another tack. |} * ''Asking the gods'': The heroes may ask the gods, or at least the Singing Rock clan wyter, where Levru and the Gagarthi are. (The Singing Rocks are the best bet, as most of the heroes have magical connections there they can call on.) Let the heroes go to town in describing the preparations, and involve the heroes in persuading the clan to support them and provide community support and objects to sacrifice. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Ask the Gods about Levru and the Gagarthi''' |- |''Appropriate abilities'': Pray to Orlanth, Pray to Ernalda, Devotee of <Deity> (-5) |- |''Modifiers and Augments'': Length of ritual (+3/hr), <Deity> Myths, various Relationships and Personality traits (for emotional involvement), various Relationships (for community support), auspicious location (e.g. the Singing Rocks) (+10), value of sacrifices (up to +10), not a member of the Singing Rock clan (-20), penalties for fatigue (contest based on time and exertion up to now). See also the Ritual Magic section, HeroQuest pp. 101-2. |- |''Resistance'': 10W3 |- |''Complete, Major, or Minor Defeat'': The heroes fail to learn anything. |- |''Marginal Defeat or Tie'': The Singing Rock wyter tells of a disturbance in the clan. |- |''Marginal Victory'': There is something wrong in the clan. Neither Levru nor the Gagarthi are touched by the wind or the earth within the Singing Rock tula. |- |''Minor Victory'': There is something wrong in the clan. Levru is within the tula; the Gagarthi are not. |- |''Major Victory'': There is discord in the Song of the Rocks, discord in the very heart of the tula. Levru is hidden within the tula. |- |''Complete Victory'': Kinstrife! Kinstrife! Kinstrife brought on by people both here and elsewhere! A vision of Levru being bound by a man with Singing Rock clan tatoos. |} By the time the rituals are complete, the heroes are liable to be too fatigued to do much more. They should go to bed disheartened that the peace deal is in danger of failure. ==== Scene II (Midpoint): When Impests Attack! ==== ''Key Points: The conspiracy is revealed when a weaponthane is driven to his death by Impests.'' At dawn the next morning, the heroes join the assembling clan atop the Singing Rocks. The mood is sombre as the clan contemplate the impending failure of the heroquest. The heroes may contemplate trying to persuade Sarostip that the quest should be abandoned, but they know that the consequences abandoning it would be worse than the consequences of failure, assuming that they could get word to Iristha nd Mikhil in time. The heroes may '''''notice that some weaponthanes are missing''' (2W): Relationship to warband, Relationship to clan (-8)'', though they won't necessarily know what that means. As the early morning worship starts, there is a scream from in the congregation as one of the weaponthanes starts to flail around wildly. Characters may '''''sense the clustering agents of reprisal''' (14): Sense gods nearby, Piety (-5)'' as the man is attacked by yavorlings and harulings (sparkers and impests) ('''Thunder Rebels''', pp. 81-82). As he attempts to flee, he runs blindly off the edge of the cliff and falls to smash on the rocks far below. Everyone is shocked, both because of the sudden and violent death, and for the calamitous omen it represents. The agents of reprisal follow him down and circle his corpse. They then rise again to swirl around the huddled clan. Perhaps their work is not yet done... ==== Scene III: The Conspiracy Uncovered ==== ''Key Points: The heroes discover that the conspiracy is led by the weaponthanes.'' All the Singing Rock heroes will recognise the dead man as Willem, a new weaponthane. They are likely to want to have pointed words with his colleagues, particularly in light of their suspicions that there is a traitor in the clan. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate the Weaponthanes''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Modifiers and Augments'': Threats of violence (-20 to +20, depending on contest v. resistance of 15W), promises of amnesty (as threats of violence), Relationship to warriors. |- |''Resistance'': 5W2 |- |''Complete or Major Defeat'': The militia manage to place the blame on the few Hiord clansmen present and incite a lynch mob. |- |''Other Defeat or Tie'': The weaponthanes reveal nothing. |- |''Marginal Victory'': Someone says that the weaponthanes have been plotting something, but there are no more details. |- |''Minor Victory'': A warrior with a mouthful of dirt blurts out that “They forced me to keep quiet!” and reveals that some weaponthanes (inlcuding Gringle, if he's an NPC) want the feud to continue. |- |''Major Victory'': The weaponthanes reveal that some of them have decided the feud should continue, and are conspiring to ensure that it does. |- |''Complete Victory'': The weaponthanes tell everything, including the involvement of Sarotar and Gringle, then attack them! |} ==== Scene IV (Plot Turn II): To the Training Ground ==== ''Key Points: The heroes rescue Levru and enter the Hero Plane.'' The heroes will either have evidence that the weaponthanes are involved, or suspect that this is the case. In either case, they are likely to want to pay a visit to their training ground, a mile or so from the stead. The training ground is a small clearing, about a hundred yards across, on a level piece of ground in the hills above the Singing Rocks. Around the field are archery butts, straw-stuffed dummies for sword practice, and other such paraphernalia. When the heroes arrive, the field is deserted. One one side of the field is the weaponthanes' longhouse. It is silent and dark, though smoke is rising from the smoke hole. The only doors are in the middle of the long side facing the field; they are closed and barred from the inside. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Play up the paranoia while the heroes are moving into the longhouse. They shouldn't know the place is deserted until they rush the longhouse. |} When the heroes enter the longhouse, three things are immediately apparent. First is the bullock in the middle of the floor, lying in a pool if its own blood. Second is Levru, gagged and tied to a chair at one end of the longhouse. Third is a large portal to the Hero Plane at the other end of the longhouse. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The portal looks rather like the Stargate from the TV series, if that would help your players visualise it. |} The bullock has been ritually slaughtered as part of a ceremony. Any god-talker will be able to deduce that the ritual was probably the one to open the portal. Levru is unharmed and will tell the heroes that, about an hour ago, a group of four weaponthanes opened the portal. They were talking about ambushing Iristha and Mikhil on the Hero Plane. Levru will tell the heroes that she recognised two of the weaponthanes as being Hiord clansmen; the remainder had Singing Rock clan tattoos. The heroes should realise that they are the only ones who can save the peace treaty, by eliminating the weaponthanes so that Iristha and Mikhil can complete their hero quest. The heroes will have to enter the hero plane. === Act III: Resolution === ==== Scene I: Rumble on the Hero Plane ==== ''Key Points: The heroes discover the full extent of the conspiracy, and it is either defeated or it isn't.'' When the heroes enter the hero plane, they can easily track the weaponthanes through a small patch of forest to where they are lying hidden above a trail, waiting to ambush Iristha and Mikhil when they pass through into the Big Teeths' lands. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Emphasise the hyper-realistic nature of the hero plane. Colours are more vibrant; vision is more acute; smells are more intense; sounds are more moving. |} The conspirators, Baranwulf, Illig, Kagrad, and Verlain (Ranged attack 16W, Close attack 6W2, Resist Magic 14W, Hardened Consiprator 15W, all after augments) will attack the heroes if attacked themselves, or if they are being beaten in a battle of wits and words. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The hero plane is the world of myth, and you should bring this out in your desciptions. When the conspirators attack, they are taking the part of the Long Teeth; their leather armour regrows stiff, spotted fur; it's difficult to hear words in their animalistic snarling; in the confusion of melee, are they attacking with their swords or their foot-long canines? Heroes, particularly Dorasa and Estave, trumpet and bellow and crush their enemies beneath their feet. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The normal physical rules don't entirely apply on the hero plane. If a character explicitly invokes a myth to justify their actions, give them a bonus of +5 or so. If they continue to invoke the same myth successfully, increase the bonus or allow them to decrease their opponents' bonus. The same bonus applies to everyone else who invokes the same myth. Some mythlets are in the appendix; the conspirators are invoking the “Big Teeth kill the Intruders” myth. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Just as Earth defeats Storm in the elemental progression, so Earth cultists have an advantage over Storm cultists in the hero plane. Give Dorasa and Estave a bonus of +5 to resist all actions by an Orlanth cultist, rising to +10 when resisting magic with magic (alternatively, use the rules from Masters of Luck and Death, p. 52) |} At some point in this confrontation, if not before, the players of Gringle and Sarotar will want to reveal their true objectives and stab the other heroes in the back. Let them. The whole episode has been building up to this struggle between the player heroes, and its is their actions that will determine whether the peace process succeeds or not. ==== Scene II: Denouement ==== If the conspirators are victorious, they move on to ambush and kill Iristha and Mikhil. Normal life in the two clans is shattered as open, total war breaks out between them. The weaponthanes gain power and prestige, but at a dreadful cost. If the heroes are successful, they hear the sounds of Issaries and Lhankor Mhy passing below just as the conspirators are defeated. The heroes can return to the Singing Rocks in time for the wedding between Askos and Levru. If the heroes succeed, but Askos cannot or will not marry Levru, another bachelor can be found but the resulting peace treaty will be neither as strong nor as long-lasting. == Acknowledgements == I'd like to thank Rob Davies for his feedback from running and playtesting this scenario. I'd also like to thank Tim Ellis, Newt Newport, and Paul Kay for their insightful reviews and comments. == Appendix: Mythlets == The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll need a copy of this myth as a handout for play. === Orlanth Banishes The Thieves === One day, Orlanth came home from a day's work in the fields to find that his favourite neck torc was missing. He looked all over his stead for it: he lifted up the mattresses, he flew up into the rafters, he swirled into all the corners of the longhouse, but he couldn't find his neck torc anywhere. Later, at dinner, he heard Vadrus, his brother, boasting to his friends in the corner of the longhouse. Orlanth left his dinner to see what Vadrus was going on about this time. When he got there, he saw that Vadrus was wearing Orlanth's favourite neck torc! "That’s mine!" shouted Orlanth. "Give it back." "No," said Vadrus. "I ... found it, in the field. It’s mine now." "But my laws say that even when something is lost, it still belongs to its owner. It’s mine." "I claim it by right of conquest!" bellowed Vadrus. "If you want it, come an take it!" But instead of reaching for his sword, Orlanth reached for his lawstaff. "This is my law," he said, "and you will obey it!" And with that he cracked Vadrus's head with the staff and Vadrus dropped the torc. Vadrus and his friends ran out of the longhouse. Orlanth chased them all the way to the edge of his tula and told them never to come back. === Humakt Slays the Enemies === After Humakt had reclaimed Death, and after Orlanth had given him a place in his hall, Humakt faced many enemies. Whenever the Sky Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Night Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Plant Tribe attacked, Humakt killed their warriors and drove them back. But when others in the Storm Tribe attacked, no-one would fight them. Even thought the Storm Tribe were enemies, they were still kin and Orlanth would not attack them. Vadrus, and Valind, and even Ragnaglar came to Orlanth's tula to fight and steal, but Orlanth would not risk kinstrife by driving them off. Humakt said that he had no kin, and so could attack the enemies, but Orlanth always told him to stay in the longhouse. Humakt worried about this. He had given his word to Orlanth always to obey his commands, but he had also given his word to Orlanth always to defend his tula. Now his two oaths were in conflict. Humakt thought long and hard about this, and then decided what to do. Soon, the winds started to blow hot and cold and fierce. It was an omen that the bad members of the Storm Tribe were coming to attack again. As soon as he felt the wind, Humakt left Orlanth’s hall. That way, when Orlanth gave the orders not to attack Valind, Humakt wasn’t there to hear them. Humakt was free to carry out his oath to defend Orlanth’s tula and, when Valind stepped onto the tula, Humakt stepped up to him and killed him. === Ernalda Heals the Tribe === Even after Ernalda told the Orlanth to make the Storm Tribe, there were problems. Even after Ernalda told Orlanth what laws to make, there were problems. Even after Ernalda told Orlanth how to judge the lawsuits, there were problems. Ernalda sat and thought about the problems and what she could do about them. She discovered that the problems were always worse just after Issaries had visited, so the next time he came, she watched very carefully. She saw that Issaries had many beautiful and wonderful things he had found in foreign lands. She saw that everyone in the Tribe wanted all the things, but Issaries wouldn’t just give them away. No, Issaries traded for them, so that everyone had to give something up to get something new. This made a lot of people unhappy, because they weren't any better off after Issaries had been than they were before, and now they were angry that there were lots of things that Issaries had that they didn’t, and that some people had more things than others. Ernalda knew what to do. One night, when everyone was sleeping, she took away all the things that Issaries had brought and hid them in her secret place. The next morning when everyone woke up, they couldn't find the things Issaries had brought and they were all very angry. They all came to Orlanth’s hall and demanded that he did something about it. Then Ernalda spoke to them and showed them that they still had the most important thing of all, and that was the Storm Tribe. Everyone realised that Ernalda was speaking the truth, and they were all happy again. Then Ernalda took everything out of her secret place, and everyone agreed to share them among themselves. == Characters == [[Peace Process characters|Characters for this episode]] are on a separate page. If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. [[Category:HeroQuest|Peace Process]] [[Category:Glorantha|Peace Process]] [[Category:Scenarios|Peace Process]] 4c807885ec5770a63722fcdb5e647f2c3ce0cf1f 1881 1874 2009-01-08T21:42:53Z Neil 2 /* Scene I: The Gagarthi Raid */ Added map of the tulas wikitext text/x-wiki <blockquote>A [[:Category:HeroQuest|HeroQuest]] Episode by Neil Smith</blockquote> == Introduction == This is an episode for [[:Category:HeroQuest|HeroQuest]]. It was written to be played by five or six players (four at a pinch) in a single three to four hour session, typical of convention games. Six [[Peace Process characters|pre-generated characters]] are provided. It is designed as a drama, rather than a thriller: the engine of the scenario lies in the characters, their motivations, and the conflicts between them. The focus should be not so much on the events that unfold during the scenario as on how the heroes react and interact as a result of those events. I advise you to understand the inter-hero tensions fully before you run the scenario. Also, when running this episode, ensure that you give plenty of time for the players to chew the scenery and explore their heroes. This episode is set in an anonymous valley in the Quivin Mountains in Sartar. All the heroes are typical Orlanthi from two neighbouring clans as they attempt to bring an end to the feud that has damaged them for decades. This episode requires the '''HeroQuest''' core rulebook. '''Thunder Rebels''' and '''Storm Tribe''' are both extremely useful for background information. The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll probably need a copy of this myth as a handout for play. I'm told that '''Barbarian Adventures''' may come in handy, but I don't actually own it! {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Many contests are described in a short form, as '''''task''' (resistance): typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. |} == Backstory == {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Read this to the players at the start of the game. Don't mention that Sarotar and Gringle are conspiring against the other heroes. |} Until twenty years ago, the neighbouring Singing Rock and Hiord clans got on tolerably well; well enough for the young Singing Rock woman, '''Estave''', to marry '''Sarotar''', her true love, and leave her birth clan to join him in the Hiord clan. Soon after that, relations between the two clans began to break down and the usual cycle of cattle raids became more vicious on both sides. Eventually, a feud was declared and the fyrds skirmished. In that first battle, Brol the farmer was killed. '''Dorasa''', his wife, managed to have him resurrected but he had the Relife sickness. Brol turned to Humakt and became '''Mortality''', Champion of the Singing Rock clan. Since then, the feud has rumbled on, sometimes as a "cold war," sometimes erupting into violence. Recently, Estave and Dorasa have managed to broker a deal that will lead the two clans to peace. The two clans have agreed to end the feud and have charged Iristha (an Issarian from the Singing Rock clan) and Mikhil (a Lhankor Mhyite from the Hiord clan) to re-enact the "Issaries the Conciliator" myth. The quest will start in the Hirod tula tomorrow morning and finish atop the Singing Rocks tomorrow evening. The purpose of the quest is to discover the terms that will end the feud. When the heroquesters return, they will bring the resolution with them; both clans have promised to honour the terms on their return. The restoration of normal relations will be cemented by the marriage of '''Askos''' and Levru, a marriage arranged by Estave. Both clans have committed all their magical resources to this quest and, if either the quest or the marriage fail to happen tomorrow, the damage to both clans will be dreadful. The heroes have been charged with ensuring that the marriage takes place tomorrow evening, atop the Singing Rocks, when the heroquesters arrive. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |But all is not well. Sarotar and '''Gringle''', warleaders of the two opposing clans, have conspired to derail the quest by kidnapping Levru and ambushing the heroquesters on the Hero Plane. They each have their own reasons for this, but both agree that war is better than peace. |} == Summary == When the episode starts, all the heroes are at the edge of the Singing Rock tula, waiting for Levru to arrive. They are to form the honour guard for Levru as she travels to her new home there. Just as she arrives, her party is ambushed by a band of Gagarthi bandits (arranged by Sarotar), who kidnap Levru. The heroes will probably track the bandits, and may or may not discover their camp. They may be able to interrogate at least one Gagarthi and learn that they were hired to perform this ambush by someone in one of the clans. The heroes may try other ways of discovering more about the Gagarthi, but to little avail. The next day, during the Windsday rituals at the Singing Rocks, one of the clan's new weaponthanes is driven wild by impests and yavorlings and falls to his death from the Rocks. The heroes should soon discover that other weaponthanes are implicated in the plot to kidnap Levru. The heroes race to the weaponthanes' training ground where they discover Levru (unharmed) and a portal to the Hero Plane. The heroes step through the portal and foil an attempt by the conspirators to ambush and kill the heroquesters Iristha and Mikhil. At some point in all this, Gringle and Sarotar reveal themselves as the plot's ringleaders, probably as they attack the other heroes. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The players are almost certainly going to move a long way from the events described here. Let them. However, the characters carry the conflict with them (the important one is between Gringle & Sarotar and the rest of the heroes), and there is a tight deadline to work to, so they should bring themselves back on track. So long as you keep that in mind, you should be able to improvise interesting events around the heroes, whatever the players have them do. As examples, in one playtest, Sarotar confessed all just after the death at the Singing Rock, shopped Gringle (an NPC in that game) and asked for a chance to put right his mistakes: a very Orlanthi thing to do. In another session, the climax was when Mortality came to fulfil his vow to kill the defeated Sarotar, only for Estave to throw herself over her husband's body: Mortality killed them both. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Don't worry too much about the pacing in this episode. The heroes are likely to move through most of it quite quickly, but you're unlikely to finish too soon. The final scene, where Gringle , Sarotar, and the other conspirators stab the other heroes in the back, will take quite a while to play out. |} == The Episode == === Act I: Introduction === ==== Scene I: The Gagarthi Raid ==== ''Key Points: The Heroes get to strut their stuff, and some Gagarthi kidnap Levru.'' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |This scene is more about giving the players a chance to give their new characters a work out than influencing the plot. |} [[Image:Peace-process-map.png|right|thumb|200px|A map of the Singing Rock and Hiord tulas]] The scene opens with the heroes standing on the edge of the Singing Rock tula, waiting for Levru to arrive. They have all just left the Singing Rock stead where they have been preparing for Levru's arrival. Mortality has witnessed the Orlanthi Greeting that gave Sarotar the Blanket gift and Estave the Meat gift; this means that the Singing Rock clan has pledged hospitality and safety to both of them ('''Thunder Rebels''', pp. 34-36). The heroes are to be Levru's honour guard as she crosses the Singing Rock tula. She is being brought across the Hiord tula by an honour guard of Hiord weaponthanes. Just as the Hiord party comes into earshot, the heroes can use their (magical) perception abilities to '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. If they succeed, describe the feel of a hot, dry wind, the sand stinging their cheek, and the smell of hot flint. As soon as they think they have been rumbled, or when the Hiord party turn up, the Gagarthi attack. Many of them attack the Hiord party; the remainder try to hold back the heroes. The heroes face one Gagarthi leader per hero and two henchmen per hero , together with the odd whirlvish or two. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Gagarthi Leader: Sword & Shield Fighting 14W, (+3 auto-augment from Tough Outlaw = 5W), Walk on Winds 6W, Stinging Winds 2W, Tough Outlaw 5W, 39AP + 27AP + 27AP (from two henchmen) = 93AP. |- |Gagarthi Henchman: Sword & Shield Fighting 5W, Stinging Winds 18, Tough Outlaw 19; treat as followers, providing 27AP each. |- |Whirlvish: Abrade 10W, Spirit combat 15, 32AP. |} If the heroes win this encounter, they drive their assailants back into the Hiord party, who then capture and escape with Levru. If the heroes lose, they are forced back and fail to prevent the Gagarthi from kidnapping Levru. Try to ensure that none of the Gagarthi leaders are captured. ==== Scene II (Plot Turn I): Kidnap and Recriminations ==== ''Key Points: The heroes discover that Levru has been taken, and that the peace deal is not a foregone conclusion.'' When the Gagarthi leave, the heroes will want to investigate the Hiord party. They will discover that the 12 Hiordlings have been defeated, many of them are injured, and they have just noticed that Levru has been taken, snatched by a wind-walking Gagarthi. Many of the Hiordlings will blame the heroes for the attack, claiming that they were on Singing Rock lands and that the Singing Rocks are responsible for the injuries. Someone might even suggest that they wanted the Gagarthi to attack (though they have no evidence to support this: it's just hot-headed talking). Tempers can be calmed in a number of ways: tending to the wounded, boasting of how the Gagarthi were defeated, vows to avenge the terrible crime. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |As the heroes take stock of the situation, reinforce that the marriage between Levru and Askos is an essential part of the peace deal. If the marriage doesn't occur at the prescribed place and time, the feud will continue at great cost to both clans. |} === Act II: Conflict === ==== Scene I: Hunting the Wild Hunters ==== ''Key Points: The heroes discover that the Gagarthi had an informer from one of the two clans.'' The ball is now in the heroes' court. They are likely to want to gain some information about the Gagarthi and Levru: where they are, why they attacked, how they knew, and what are they planning for Levru. There are a few avenues that could be open to them. There could be Gagarthi prisoners to interrogate. Some Gagarthi fled on foot and can be tracked. The god-talker heroes may want to use divinations. Also, someone will have to tell the Singing Rock clan the bad news of Levru's kidnap. * ''Interrogating Gagarthi'': The bandits are fanatics, and all declared outlaw. They know that if they are ever brought to justice they are almost certain to be executed, so they have no real reason to co-operate. This makes interrogation difficult. The heroes know this (inform the players if need by), but they are likely anyway to want to interrogate them anyway. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate Gagarthi Bandits''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Resistance'': 12W2 |- |''Complete or Major Defeat'': The bandit either commits suicide or manages to snatch a hostage. |- |''Other Defeat or Tie'': The bandit reveals nothing. |- |''Marginal Victory'': The bandit hints that they knew that someone would be here. |- |''Minor Victory'': The bandits were waiting specifically for this group of Hiord clansfolk. |- |''Major Victory'': The Gagarthi were hired to kidnap “the girl.” |- |''Complete Victory'': This was definitely an inside job, connected to some big ritual. He doesn't know the identity of the insider. |} Heroes may '''''Recognise the Bandit''' (15W): Relationship to Clan'' and recall the bandit's name and deeds, and reduce the bandit's resistance according to the result of that contest. * ''Tracking the Gagarthi'': The wind-walking Gagarthi leaders are the ones who stole Levru, and they are difficult to track. However, the other Gagarthi who escaped are easier to follow: '''''Track the Gagarthi''' (3W): Track, Scout (-5), Singing Rock Geography (-10), Hiord Geography (-10)''. Any victory allows the heroes to follow the trail. It soon gets easier to follow once the wind-walking bandits come to earth. After a few hours of hard walking over some increasingly rough terrain (up and down some some very steep and heavily wooded slopes), the heroes come to the Gagarthi's (deserted) camp. The camp has been packed up recently and the firepit it still hot. Observant heroes will notice two fresh trails leaving the camp, one more heavily used than the other. Both peter out after a mile or so as they pass over bare, rocky ground in the fading light. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Their job done, the Gagarthi have been paid off and have left the area. Levru has been taken by some weaponthanes to the training camp in the Singing Rock tula. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |One potential problem point is persuading the heroes that they've lost the Gagarthi's trail. Stress the difficulty of following the trail, the worsening conditions, and time pressure the heroes are under to force them into trying another tack. |} * ''Asking the gods'': The heroes may ask the gods, or at least the Singing Rock clan wyter, where Levru and the Gagarthi are. (The Singing Rocks are the best bet, as most of the heroes have magical connections there they can call on.) Let the heroes go to town in describing the preparations, and involve the heroes in persuading the clan to support them and provide community support and objects to sacrifice. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Ask the Gods about Levru and the Gagarthi''' |- |''Appropriate abilities'': Pray to Orlanth, Pray to Ernalda, Devotee of <Deity> (-5) |- |''Modifiers and Augments'': Length of ritual (+3/hr), <Deity> Myths, various Relationships and Personality traits (for emotional involvement), various Relationships (for community support), auspicious location (e.g. the Singing Rocks) (+10), value of sacrifices (up to +10), not a member of the Singing Rock clan (-20), penalties for fatigue (contest based on time and exertion up to now). See also the Ritual Magic section, HeroQuest pp. 101-2. |- |''Resistance'': 10W3 |- |''Complete, Major, or Minor Defeat'': The heroes fail to learn anything. |- |''Marginal Defeat or Tie'': The Singing Rock wyter tells of a disturbance in the clan. |- |''Marginal Victory'': There is something wrong in the clan. Neither Levru nor the Gagarthi are touched by the wind or the earth within the Singing Rock tula. |- |''Minor Victory'': There is something wrong in the clan. Levru is within the tula; the Gagarthi are not. |- |''Major Victory'': There is discord in the Song of the Rocks, discord in the very heart of the tula. Levru is hidden within the tula. |- |''Complete Victory'': Kinstrife! Kinstrife! Kinstrife brought on by people both here and elsewhere! A vision of Levru being bound by a man with Singing Rock clan tatoos. |} By the time the rituals are complete, the heroes are liable to be too fatigued to do much more. They should go to bed disheartened that the peace deal is in danger of failure. ==== Scene II (Midpoint): When Impests Attack! ==== ''Key Points: The conspiracy is revealed when a weaponthane is driven to his death by Impests.'' At dawn the next morning, the heroes join the assembling clan atop the Singing Rocks. The mood is sombre as the clan contemplate the impending failure of the heroquest. The heroes may contemplate trying to persuade Sarostip that the quest should be abandoned, but they know that the consequences abandoning it would be worse than the consequences of failure, assuming that they could get word to Iristha nd Mikhil in time. The heroes may '''''notice that some weaponthanes are missing''' (2W): Relationship to warband, Relationship to clan (-8)'', though they won't necessarily know what that means. As the early morning worship starts, there is a scream from in the congregation as one of the weaponthanes starts to flail around wildly. Characters may '''''sense the clustering agents of reprisal''' (14): Sense gods nearby, Piety (-5)'' as the man is attacked by yavorlings and harulings (sparkers and impests) ('''Thunder Rebels''', pp. 81-82). As he attempts to flee, he runs blindly off the edge of the cliff and falls to smash on the rocks far below. Everyone is shocked, both because of the sudden and violent death, and for the calamitous omen it represents. The agents of reprisal follow him down and circle his corpse. They then rise again to swirl around the huddled clan. Perhaps their work is not yet done... ==== Scene III: The Conspiracy Uncovered ==== ''Key Points: The heroes discover that the conspiracy is led by the weaponthanes.'' All the Singing Rock heroes will recognise the dead man as Willem, a new weaponthane. They are likely to want to have pointed words with his colleagues, particularly in light of their suspicions that there is a traitor in the clan. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate the Weaponthanes''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Modifiers and Augments'': Threats of violence (-20 to +20, depending on contest v. resistance of 15W), promises of amnesty (as threats of violence), Relationship to warriors. |- |''Resistance'': 5W2 |- |''Complete or Major Defeat'': The militia manage to place the blame on the few Hiord clansmen present and incite a lynch mob. |- |''Other Defeat or Tie'': The weaponthanes reveal nothing. |- |''Marginal Victory'': Someone says that the weaponthanes have been plotting something, but there are no more details. |- |''Minor Victory'': A warrior with a mouthful of dirt blurts out that “They forced me to keep quiet!” and reveals that some weaponthanes (inlcuding Gringle, if he's an NPC) want the feud to continue. |- |''Major Victory'': The weaponthanes reveal that some of them have decided the feud should continue, and are conspiring to ensure that it does. |- |''Complete Victory'': The weaponthanes tell everything, including the involvement of Sarotar and Gringle, then attack them! |} ==== Scene IV (Plot Turn II): To the Training Ground ==== ''Key Points: The heroes rescue Levru and enter the Hero Plane.'' The heroes will either have evidence that the weaponthanes are involved, or suspect that this is the case. In either case, they are likely to want to pay a visit to their training ground, a mile or so from the stead. The training ground is a small clearing, about a hundred yards across, on a level piece of ground in the hills above the Singing Rocks. Around the field are archery butts, straw-stuffed dummies for sword practice, and other such paraphernalia. When the heroes arrive, the field is deserted. One one side of the field is the weaponthanes' longhouse. It is silent and dark, though smoke is rising from the smoke hole. The only doors are in the middle of the long side facing the field; they are closed and barred from the inside. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Play up the paranoia while the heroes are moving into the longhouse. They shouldn't know the place is deserted until they rush the longhouse. |} When the heroes enter the longhouse, three things are immediately apparent. First is the bullock in the middle of the floor, lying in a pool if its own blood. Second is Levru, gagged and tied to a chair at one end of the longhouse. Third is a large portal to the Hero Plane at the other end of the longhouse. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The portal looks rather like the Stargate from the TV series, if that would help your players visualise it. |} The bullock has been ritually slaughtered as part of a ceremony. Any god-talker will be able to deduce that the ritual was probably the one to open the portal. Levru is unharmed and will tell the heroes that, about an hour ago, a group of four weaponthanes opened the portal. They were talking about ambushing Iristha and Mikhil on the Hero Plane. Levru will tell the heroes that she recognised two of the weaponthanes as being Hiord clansmen; the remainder had Singing Rock clan tattoos. The heroes should realise that they are the only ones who can save the peace treaty, by eliminating the weaponthanes so that Iristha and Mikhil can complete their hero quest. The heroes will have to enter the hero plane. === Act III: Resolution === ==== Scene I: Rumble on the Hero Plane ==== ''Key Points: The heroes discover the full extent of the conspiracy, and it is either defeated or it isn't.'' When the heroes enter the hero plane, they can easily track the weaponthanes through a small patch of forest to where they are lying hidden above a trail, waiting to ambush Iristha and Mikhil when they pass through into the Big Teeths' lands. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Emphasise the hyper-realistic nature of the hero plane. Colours are more vibrant; vision is more acute; smells are more intense; sounds are more moving. |} The conspirators, Baranwulf, Illig, Kagrad, and Verlain (Ranged attack 16W, Close attack 6W2, Resist Magic 14W, Hardened Consiprator 15W, all after augments) will attack the heroes if attacked themselves, or if they are being beaten in a battle of wits and words. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The hero plane is the world of myth, and you should bring this out in your desciptions. When the conspirators attack, they are taking the part of the Long Teeth; their leather armour regrows stiff, spotted fur; it's difficult to hear words in their animalistic snarling; in the confusion of melee, are they attacking with their swords or their foot-long canines? Heroes, particularly Dorasa and Estave, trumpet and bellow and crush their enemies beneath their feet. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The normal physical rules don't entirely apply on the hero plane. If a character explicitly invokes a myth to justify their actions, give them a bonus of +5 or so. If they continue to invoke the same myth successfully, increase the bonus or allow them to decrease their opponents' bonus. The same bonus applies to everyone else who invokes the same myth. Some mythlets are in the appendix; the conspirators are invoking the “Big Teeth kill the Intruders” myth. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Just as Earth defeats Storm in the elemental progression, so Earth cultists have an advantage over Storm cultists in the hero plane. Give Dorasa and Estave a bonus of +5 to resist all actions by an Orlanth cultist, rising to +10 when resisting magic with magic (alternatively, use the rules from Masters of Luck and Death, p. 52) |} At some point in this confrontation, if not before, the players of Gringle and Sarotar will want to reveal their true objectives and stab the other heroes in the back. Let them. The whole episode has been building up to this struggle between the player heroes, and its is their actions that will determine whether the peace process succeeds or not. ==== Scene II: Denouement ==== If the conspirators are victorious, they move on to ambush and kill Iristha and Mikhil. Normal life in the two clans is shattered as open, total war breaks out between them. The weaponthanes gain power and prestige, but at a dreadful cost. If the heroes are successful, they hear the sounds of Issaries and Lhankor Mhy passing below just as the conspirators are defeated. The heroes can return to the Singing Rocks in time for the wedding between Askos and Levru. If the heroes succeed, but Askos cannot or will not marry Levru, another bachelor can be found but the resulting peace treaty will be neither as strong nor as long-lasting. == Acknowledgements == I'd like to thank Rob Davies for his feedback from running and playtesting this scenario. I'd also like to thank Tim Ellis, Newt Newport, and Paul Kay for their insightful reviews and comments. == Appendix: Mythlets == The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll need a copy of this myth as a handout for play. === Orlanth Banishes The Thieves === One day, Orlanth came home from a day's work in the fields to find that his favourite neck torc was missing. He looked all over his stead for it: he lifted up the mattresses, he flew up into the rafters, he swirled into all the corners of the longhouse, but he couldn't find his neck torc anywhere. Later, at dinner, he heard Vadrus, his brother, boasting to his friends in the corner of the longhouse. Orlanth left his dinner to see what Vadrus was going on about this time. When he got there, he saw that Vadrus was wearing Orlanth's favourite neck torc! "That’s mine!" shouted Orlanth. "Give it back." "No," said Vadrus. "I ... found it, in the field. It’s mine now." "But my laws say that even when something is lost, it still belongs to its owner. It’s mine." "I claim it by right of conquest!" bellowed Vadrus. "If you want it, come an take it!" But instead of reaching for his sword, Orlanth reached for his lawstaff. "This is my law," he said, "and you will obey it!" And with that he cracked Vadrus's head with the staff and Vadrus dropped the torc. Vadrus and his friends ran out of the longhouse. Orlanth chased them all the way to the edge of his tula and told them never to come back. === Humakt Slays the Enemies === After Humakt had reclaimed Death, and after Orlanth had given him a place in his hall, Humakt faced many enemies. Whenever the Sky Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Night Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Plant Tribe attacked, Humakt killed their warriors and drove them back. But when others in the Storm Tribe attacked, no-one would fight them. Even thought the Storm Tribe were enemies, they were still kin and Orlanth would not attack them. Vadrus, and Valind, and even Ragnaglar came to Orlanth's tula to fight and steal, but Orlanth would not risk kinstrife by driving them off. Humakt said that he had no kin, and so could attack the enemies, but Orlanth always told him to stay in the longhouse. Humakt worried about this. He had given his word to Orlanth always to obey his commands, but he had also given his word to Orlanth always to defend his tula. Now his two oaths were in conflict. Humakt thought long and hard about this, and then decided what to do. Soon, the winds started to blow hot and cold and fierce. It was an omen that the bad members of the Storm Tribe were coming to attack again. As soon as he felt the wind, Humakt left Orlanth’s hall. That way, when Orlanth gave the orders not to attack Valind, Humakt wasn’t there to hear them. Humakt was free to carry out his oath to defend Orlanth’s tula and, when Valind stepped onto the tula, Humakt stepped up to him and killed him. === Ernalda Heals the Tribe === Even after Ernalda told the Orlanth to make the Storm Tribe, there were problems. Even after Ernalda told Orlanth what laws to make, there were problems. Even after Ernalda told Orlanth how to judge the lawsuits, there were problems. Ernalda sat and thought about the problems and what she could do about them. She discovered that the problems were always worse just after Issaries had visited, so the next time he came, she watched very carefully. She saw that Issaries had many beautiful and wonderful things he had found in foreign lands. She saw that everyone in the Tribe wanted all the things, but Issaries wouldn’t just give them away. No, Issaries traded for them, so that everyone had to give something up to get something new. This made a lot of people unhappy, because they weren't any better off after Issaries had been than they were before, and now they were angry that there were lots of things that Issaries had that they didn’t, and that some people had more things than others. Ernalda knew what to do. One night, when everyone was sleeping, she took away all the things that Issaries had brought and hid them in her secret place. The next morning when everyone woke up, they couldn't find the things Issaries had brought and they were all very angry. They all came to Orlanth’s hall and demanded that he did something about it. Then Ernalda spoke to them and showed them that they still had the most important thing of all, and that was the Storm Tribe. Everyone realised that Ernalda was speaking the truth, and they were all happy again. Then Ernalda took everything out of her secret place, and everyone agreed to share them among themselves. == Characters == [[Peace Process characters|Characters for this episode]] are on a separate page. If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. [[Category:HeroQuest|Peace Process]] [[Category:Glorantha|Peace Process]] [[Category:Scenarios|Peace Process]] 2b8cb1eec8867f11c6aae21bb05b699fa9b9cfd5 1883 1881 2009-01-08T21:50:06Z Neil 2 /* Scene I: The Gagarthi Raid */ wikitext text/x-wiki <blockquote>A [[:Category:HeroQuest|HeroQuest]] Episode by Neil Smith</blockquote> == Introduction == This is an episode for [[:Category:HeroQuest|HeroQuest]]. It was written to be played by five or six players (four at a pinch) in a single three to four hour session, typical of convention games. Six [[Peace Process characters|pre-generated characters]] are provided. It is designed as a drama, rather than a thriller: the engine of the scenario lies in the characters, their motivations, and the conflicts between them. The focus should be not so much on the events that unfold during the scenario as on how the heroes react and interact as a result of those events. I advise you to understand the inter-hero tensions fully before you run the scenario. Also, when running this episode, ensure that you give plenty of time for the players to chew the scenery and explore their heroes. This episode is set in an anonymous valley in the Quivin Mountains in Sartar. All the heroes are typical Orlanthi from two neighbouring clans as they attempt to bring an end to the feud that has damaged them for decades. This episode requires the '''HeroQuest''' core rulebook. '''Thunder Rebels''' and '''Storm Tribe''' are both extremely useful for background information. The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll probably need a copy of this myth as a handout for play. I'm told that '''Barbarian Adventures''' may come in handy, but I don't actually own it! {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Many contests are described in a short form, as '''''task''' (resistance): typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. |} == Backstory == {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Read this to the players at the start of the game. Don't mention that Sarotar and Gringle are conspiring against the other heroes. |} Until twenty years ago, the neighbouring Singing Rock and Hiord clans got on tolerably well; well enough for the young Singing Rock woman, '''Estave''', to marry '''Sarotar''', her true love, and leave her birth clan to join him in the Hiord clan. Soon after that, relations between the two clans began to break down and the usual cycle of cattle raids became more vicious on both sides. Eventually, a feud was declared and the fyrds skirmished. In that first battle, Brol the farmer was killed. '''Dorasa''', his wife, managed to have him resurrected but he had the Relife sickness. Brol turned to Humakt and became '''Mortality''', Champion of the Singing Rock clan. Since then, the feud has rumbled on, sometimes as a "cold war," sometimes erupting into violence. Recently, Estave and Dorasa have managed to broker a deal that will lead the two clans to peace. The two clans have agreed to end the feud and have charged Iristha (an Issarian from the Singing Rock clan) and Mikhil (a Lhankor Mhyite from the Hiord clan) to re-enact the "Issaries the Conciliator" myth. The quest will start in the Hirod tula tomorrow morning and finish atop the Singing Rocks tomorrow evening. The purpose of the quest is to discover the terms that will end the feud. When the heroquesters return, they will bring the resolution with them; both clans have promised to honour the terms on their return. The restoration of normal relations will be cemented by the marriage of '''Askos''' and Levru, a marriage arranged by Estave. Both clans have committed all their magical resources to this quest and, if either the quest or the marriage fail to happen tomorrow, the damage to both clans will be dreadful. The heroes have been charged with ensuring that the marriage takes place tomorrow evening, atop the Singing Rocks, when the heroquesters arrive. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |But all is not well. Sarotar and '''Gringle''', warleaders of the two opposing clans, have conspired to derail the quest by kidnapping Levru and ambushing the heroquesters on the Hero Plane. They each have their own reasons for this, but both agree that war is better than peace. |} == Summary == When the episode starts, all the heroes are at the edge of the Singing Rock tula, waiting for Levru to arrive. They are to form the honour guard for Levru as she travels to her new home there. Just as she arrives, her party is ambushed by a band of Gagarthi bandits (arranged by Sarotar), who kidnap Levru. The heroes will probably track the bandits, and may or may not discover their camp. They may be able to interrogate at least one Gagarthi and learn that they were hired to perform this ambush by someone in one of the clans. The heroes may try other ways of discovering more about the Gagarthi, but to little avail. The next day, during the Windsday rituals at the Singing Rocks, one of the clan's new weaponthanes is driven wild by impests and yavorlings and falls to his death from the Rocks. The heroes should soon discover that other weaponthanes are implicated in the plot to kidnap Levru. The heroes race to the weaponthanes' training ground where they discover Levru (unharmed) and a portal to the Hero Plane. The heroes step through the portal and foil an attempt by the conspirators to ambush and kill the heroquesters Iristha and Mikhil. At some point in all this, Gringle and Sarotar reveal themselves as the plot's ringleaders, probably as they attack the other heroes. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The players are almost certainly going to move a long way from the events described here. Let them. However, the characters carry the conflict with them (the important one is between Gringle & Sarotar and the rest of the heroes), and there is a tight deadline to work to, so they should bring themselves back on track. So long as you keep that in mind, you should be able to improvise interesting events around the heroes, whatever the players have them do. As examples, in one playtest, Sarotar confessed all just after the death at the Singing Rock, shopped Gringle (an NPC in that game) and asked for a chance to put right his mistakes: a very Orlanthi thing to do. In another session, the climax was when Mortality came to fulfil his vow to kill the defeated Sarotar, only for Estave to throw herself over her husband's body: Mortality killed them both. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Don't worry too much about the pacing in this episode. The heroes are likely to move through most of it quite quickly, but you're unlikely to finish too soon. The final scene, where Gringle , Sarotar, and the other conspirators stab the other heroes in the back, will take quite a while to play out. |} == The Episode == === Act I: Introduction === ==== Scene I: The Gagarthi Raid ==== ''Key Points: The Heroes get to strut their stuff, and some Gagarthi kidnap Levru.'' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |This scene is more about giving the players a chance to give their new characters a work out than influencing the plot. |} [[Image:Peace-process-map.png|right|thumb|200px|A map of the Singing Rock and Hiord tulas. This is an approximate map of the various locations of significance in this episode. The two tulas are in neighbouring valleys in the Quivin mountains. The land between them is mountainous and heavily forested, suitable only for hunting. The nearest the two tulas come to each other is at a pass between the high sheep pastures. ]] The scene opens with the heroes standing on the edge of the Singing Rock tula, waiting for Levru to arrive. They have all just left the Singing Rock stead where they have been preparing for Levru's arrival. Mortality has witnessed the Orlanthi Greeting that gave Sarotar the Blanket gift and Estave the Meat gift; this means that the Singing Rock clan has pledged hospitality and safety to both of them ('''Thunder Rebels''', pp. 34-36). The heroes are to be Levru's honour guard as she crosses the Singing Rock tula. She is being brought across the Hiord tula by an honour guard of Hiord weaponthanes. Just as the Hiord party comes into earshot, the heroes can use their (magical) perception abilities to '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. If they succeed, describe the feel of a hot, dry wind, the sand stinging their cheek, and the smell of hot flint. As soon as they think they have been rumbled, or when the Hiord party turn up, the Gagarthi attack. Many of them attack the Hiord party; the remainder try to hold back the heroes. The heroes face one Gagarthi leader per hero and two henchmen per hero , together with the odd whirlvish or two. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Gagarthi Leader: Sword & Shield Fighting 14W, (+3 auto-augment from Tough Outlaw = 5W), Walk on Winds 6W, Stinging Winds 2W, Tough Outlaw 5W, 39AP + 27AP + 27AP (from two henchmen) = 93AP. |- |Gagarthi Henchman: Sword & Shield Fighting 5W, Stinging Winds 18, Tough Outlaw 19; treat as followers, providing 27AP each. |- |Whirlvish: Abrade 10W, Spirit combat 15, 32AP. |} If the heroes win this encounter, they drive their assailants back into the Hiord party, who then capture and escape with Levru. If the heroes lose, they are forced back and fail to prevent the Gagarthi from kidnapping Levru. Try to ensure that none of the Gagarthi leaders are captured. ==== Scene II (Plot Turn I): Kidnap and Recriminations ==== ''Key Points: The heroes discover that Levru has been taken, and that the peace deal is not a foregone conclusion.'' When the Gagarthi leave, the heroes will want to investigate the Hiord party. They will discover that the 12 Hiordlings have been defeated, many of them are injured, and they have just noticed that Levru has been taken, snatched by a wind-walking Gagarthi. Many of the Hiordlings will blame the heroes for the attack, claiming that they were on Singing Rock lands and that the Singing Rocks are responsible for the injuries. Someone might even suggest that they wanted the Gagarthi to attack (though they have no evidence to support this: it's just hot-headed talking). Tempers can be calmed in a number of ways: tending to the wounded, boasting of how the Gagarthi were defeated, vows to avenge the terrible crime. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |As the heroes take stock of the situation, reinforce that the marriage between Levru and Askos is an essential part of the peace deal. If the marriage doesn't occur at the prescribed place and time, the feud will continue at great cost to both clans. |} === Act II: Conflict === ==== Scene I: Hunting the Wild Hunters ==== ''Key Points: The heroes discover that the Gagarthi had an informer from one of the two clans.'' The ball is now in the heroes' court. They are likely to want to gain some information about the Gagarthi and Levru: where they are, why they attacked, how they knew, and what are they planning for Levru. There are a few avenues that could be open to them. There could be Gagarthi prisoners to interrogate. Some Gagarthi fled on foot and can be tracked. The god-talker heroes may want to use divinations. Also, someone will have to tell the Singing Rock clan the bad news of Levru's kidnap. * ''Interrogating Gagarthi'': The bandits are fanatics, and all declared outlaw. They know that if they are ever brought to justice they are almost certain to be executed, so they have no real reason to co-operate. This makes interrogation difficult. The heroes know this (inform the players if need by), but they are likely anyway to want to interrogate them anyway. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate Gagarthi Bandits''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Resistance'': 12W2 |- |''Complete or Major Defeat'': The bandit either commits suicide or manages to snatch a hostage. |- |''Other Defeat or Tie'': The bandit reveals nothing. |- |''Marginal Victory'': The bandit hints that they knew that someone would be here. |- |''Minor Victory'': The bandits were waiting specifically for this group of Hiord clansfolk. |- |''Major Victory'': The Gagarthi were hired to kidnap “the girl.” |- |''Complete Victory'': This was definitely an inside job, connected to some big ritual. He doesn't know the identity of the insider. |} Heroes may '''''Recognise the Bandit''' (15W): Relationship to Clan'' and recall the bandit's name and deeds, and reduce the bandit's resistance according to the result of that contest. * ''Tracking the Gagarthi'': The wind-walking Gagarthi leaders are the ones who stole Levru, and they are difficult to track. However, the other Gagarthi who escaped are easier to follow: '''''Track the Gagarthi''' (3W): Track, Scout (-5), Singing Rock Geography (-10), Hiord Geography (-10)''. Any victory allows the heroes to follow the trail. It soon gets easier to follow once the wind-walking bandits come to earth. After a few hours of hard walking over some increasingly rough terrain (up and down some some very steep and heavily wooded slopes), the heroes come to the Gagarthi's (deserted) camp. The camp has been packed up recently and the firepit it still hot. Observant heroes will notice two fresh trails leaving the camp, one more heavily used than the other. Both peter out after a mile or so as they pass over bare, rocky ground in the fading light. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Their job done, the Gagarthi have been paid off and have left the area. Levru has been taken by some weaponthanes to the training camp in the Singing Rock tula. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |One potential problem point is persuading the heroes that they've lost the Gagarthi's trail. Stress the difficulty of following the trail, the worsening conditions, and time pressure the heroes are under to force them into trying another tack. |} * ''Asking the gods'': The heroes may ask the gods, or at least the Singing Rock clan wyter, where Levru and the Gagarthi are. (The Singing Rocks are the best bet, as most of the heroes have magical connections there they can call on.) Let the heroes go to town in describing the preparations, and involve the heroes in persuading the clan to support them and provide community support and objects to sacrifice. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Ask the Gods about Levru and the Gagarthi''' |- |''Appropriate abilities'': Pray to Orlanth, Pray to Ernalda, Devotee of <Deity> (-5) |- |''Modifiers and Augments'': Length of ritual (+3/hr), <Deity> Myths, various Relationships and Personality traits (for emotional involvement), various Relationships (for community support), auspicious location (e.g. the Singing Rocks) (+10), value of sacrifices (up to +10), not a member of the Singing Rock clan (-20), penalties for fatigue (contest based on time and exertion up to now). See also the Ritual Magic section, HeroQuest pp. 101-2. |- |''Resistance'': 10W3 |- |''Complete, Major, or Minor Defeat'': The heroes fail to learn anything. |- |''Marginal Defeat or Tie'': The Singing Rock wyter tells of a disturbance in the clan. |- |''Marginal Victory'': There is something wrong in the clan. Neither Levru nor the Gagarthi are touched by the wind or the earth within the Singing Rock tula. |- |''Minor Victory'': There is something wrong in the clan. Levru is within the tula; the Gagarthi are not. |- |''Major Victory'': There is discord in the Song of the Rocks, discord in the very heart of the tula. Levru is hidden within the tula. |- |''Complete Victory'': Kinstrife! Kinstrife! Kinstrife brought on by people both here and elsewhere! A vision of Levru being bound by a man with Singing Rock clan tatoos. |} By the time the rituals are complete, the heroes are liable to be too fatigued to do much more. They should go to bed disheartened that the peace deal is in danger of failure. ==== Scene II (Midpoint): When Impests Attack! ==== ''Key Points: The conspiracy is revealed when a weaponthane is driven to his death by Impests.'' At dawn the next morning, the heroes join the assembling clan atop the Singing Rocks. The mood is sombre as the clan contemplate the impending failure of the heroquest. The heroes may contemplate trying to persuade Sarostip that the quest should be abandoned, but they know that the consequences abandoning it would be worse than the consequences of failure, assuming that they could get word to Iristha nd Mikhil in time. The heroes may '''''notice that some weaponthanes are missing''' (2W): Relationship to warband, Relationship to clan (-8)'', though they won't necessarily know what that means. As the early morning worship starts, there is a scream from in the congregation as one of the weaponthanes starts to flail around wildly. Characters may '''''sense the clustering agents of reprisal''' (14): Sense gods nearby, Piety (-5)'' as the man is attacked by yavorlings and harulings (sparkers and impests) ('''Thunder Rebels''', pp. 81-82). As he attempts to flee, he runs blindly off the edge of the cliff and falls to smash on the rocks far below. Everyone is shocked, both because of the sudden and violent death, and for the calamitous omen it represents. The agents of reprisal follow him down and circle his corpse. They then rise again to swirl around the huddled clan. Perhaps their work is not yet done... ==== Scene III: The Conspiracy Uncovered ==== ''Key Points: The heroes discover that the conspiracy is led by the weaponthanes.'' All the Singing Rock heroes will recognise the dead man as Willem, a new weaponthane. They are likely to want to have pointed words with his colleagues, particularly in light of their suspicions that there is a traitor in the clan. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate the Weaponthanes''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Modifiers and Augments'': Threats of violence (-20 to +20, depending on contest v. resistance of 15W), promises of amnesty (as threats of violence), Relationship to warriors. |- |''Resistance'': 5W2 |- |''Complete or Major Defeat'': The militia manage to place the blame on the few Hiord clansmen present and incite a lynch mob. |- |''Other Defeat or Tie'': The weaponthanes reveal nothing. |- |''Marginal Victory'': Someone says that the weaponthanes have been plotting something, but there are no more details. |- |''Minor Victory'': A warrior with a mouthful of dirt blurts out that “They forced me to keep quiet!” and reveals that some weaponthanes (inlcuding Gringle, if he's an NPC) want the feud to continue. |- |''Major Victory'': The weaponthanes reveal that some of them have decided the feud should continue, and are conspiring to ensure that it does. |- |''Complete Victory'': The weaponthanes tell everything, including the involvement of Sarotar and Gringle, then attack them! |} ==== Scene IV (Plot Turn II): To the Training Ground ==== ''Key Points: The heroes rescue Levru and enter the Hero Plane.'' The heroes will either have evidence that the weaponthanes are involved, or suspect that this is the case. In either case, they are likely to want to pay a visit to their training ground, a mile or so from the stead. The training ground is a small clearing, about a hundred yards across, on a level piece of ground in the hills above the Singing Rocks. Around the field are archery butts, straw-stuffed dummies for sword practice, and other such paraphernalia. When the heroes arrive, the field is deserted. One one side of the field is the weaponthanes' longhouse. It is silent and dark, though smoke is rising from the smoke hole. The only doors are in the middle of the long side facing the field; they are closed and barred from the inside. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Play up the paranoia while the heroes are moving into the longhouse. They shouldn't know the place is deserted until they rush the longhouse. |} When the heroes enter the longhouse, three things are immediately apparent. First is the bullock in the middle of the floor, lying in a pool if its own blood. Second is Levru, gagged and tied to a chair at one end of the longhouse. Third is a large portal to the Hero Plane at the other end of the longhouse. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The portal looks rather like the Stargate from the TV series, if that would help your players visualise it. |} The bullock has been ritually slaughtered as part of a ceremony. Any god-talker will be able to deduce that the ritual was probably the one to open the portal. Levru is unharmed and will tell the heroes that, about an hour ago, a group of four weaponthanes opened the portal. They were talking about ambushing Iristha and Mikhil on the Hero Plane. Levru will tell the heroes that she recognised two of the weaponthanes as being Hiord clansmen; the remainder had Singing Rock clan tattoos. The heroes should realise that they are the only ones who can save the peace treaty, by eliminating the weaponthanes so that Iristha and Mikhil can complete their hero quest. The heroes will have to enter the hero plane. === Act III: Resolution === ==== Scene I: Rumble on the Hero Plane ==== ''Key Points: The heroes discover the full extent of the conspiracy, and it is either defeated or it isn't.'' When the heroes enter the hero plane, they can easily track the weaponthanes through a small patch of forest to where they are lying hidden above a trail, waiting to ambush Iristha and Mikhil when they pass through into the Big Teeths' lands. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Emphasise the hyper-realistic nature of the hero plane. Colours are more vibrant; vision is more acute; smells are more intense; sounds are more moving. |} The conspirators, Baranwulf, Illig, Kagrad, and Verlain (Ranged attack 16W, Close attack 6W2, Resist Magic 14W, Hardened Consiprator 15W, all after augments) will attack the heroes if attacked themselves, or if they are being beaten in a battle of wits and words. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The hero plane is the world of myth, and you should bring this out in your desciptions. When the conspirators attack, they are taking the part of the Long Teeth; their leather armour regrows stiff, spotted fur; it's difficult to hear words in their animalistic snarling; in the confusion of melee, are they attacking with their swords or their foot-long canines? Heroes, particularly Dorasa and Estave, trumpet and bellow and crush their enemies beneath their feet. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The normal physical rules don't entirely apply on the hero plane. If a character explicitly invokes a myth to justify their actions, give them a bonus of +5 or so. If they continue to invoke the same myth successfully, increase the bonus or allow them to decrease their opponents' bonus. The same bonus applies to everyone else who invokes the same myth. Some mythlets are in the appendix; the conspirators are invoking the “Big Teeth kill the Intruders” myth. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Just as Earth defeats Storm in the elemental progression, so Earth cultists have an advantage over Storm cultists in the hero plane. Give Dorasa and Estave a bonus of +5 to resist all actions by an Orlanth cultist, rising to +10 when resisting magic with magic (alternatively, use the rules from Masters of Luck and Death, p. 52) |} At some point in this confrontation, if not before, the players of Gringle and Sarotar will want to reveal their true objectives and stab the other heroes in the back. Let them. The whole episode has been building up to this struggle between the player heroes, and its is their actions that will determine whether the peace process succeeds or not. ==== Scene II: Denouement ==== If the conspirators are victorious, they move on to ambush and kill Iristha and Mikhil. Normal life in the two clans is shattered as open, total war breaks out between them. The weaponthanes gain power and prestige, but at a dreadful cost. If the heroes are successful, they hear the sounds of Issaries and Lhankor Mhy passing below just as the conspirators are defeated. The heroes can return to the Singing Rocks in time for the wedding between Askos and Levru. If the heroes succeed, but Askos cannot or will not marry Levru, another bachelor can be found but the resulting peace treaty will be neither as strong nor as long-lasting. == Acknowledgements == I'd like to thank Rob Davies for his feedback from running and playtesting this scenario. I'd also like to thank Tim Ellis, Newt Newport, and Paul Kay for their insightful reviews and comments. == Appendix: Mythlets == The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll need a copy of this myth as a handout for play. === Orlanth Banishes The Thieves === One day, Orlanth came home from a day's work in the fields to find that his favourite neck torc was missing. He looked all over his stead for it: he lifted up the mattresses, he flew up into the rafters, he swirled into all the corners of the longhouse, but he couldn't find his neck torc anywhere. Later, at dinner, he heard Vadrus, his brother, boasting to his friends in the corner of the longhouse. Orlanth left his dinner to see what Vadrus was going on about this time. When he got there, he saw that Vadrus was wearing Orlanth's favourite neck torc! "That’s mine!" shouted Orlanth. "Give it back." "No," said Vadrus. "I ... found it, in the field. It’s mine now." "But my laws say that even when something is lost, it still belongs to its owner. It’s mine." "I claim it by right of conquest!" bellowed Vadrus. "If you want it, come an take it!" But instead of reaching for his sword, Orlanth reached for his lawstaff. "This is my law," he said, "and you will obey it!" And with that he cracked Vadrus's head with the staff and Vadrus dropped the torc. Vadrus and his friends ran out of the longhouse. Orlanth chased them all the way to the edge of his tula and told them never to come back. === Humakt Slays the Enemies === After Humakt had reclaimed Death, and after Orlanth had given him a place in his hall, Humakt faced many enemies. Whenever the Sky Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Night Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Plant Tribe attacked, Humakt killed their warriors and drove them back. But when others in the Storm Tribe attacked, no-one would fight them. Even thought the Storm Tribe were enemies, they were still kin and Orlanth would not attack them. Vadrus, and Valind, and even Ragnaglar came to Orlanth's tula to fight and steal, but Orlanth would not risk kinstrife by driving them off. Humakt said that he had no kin, and so could attack the enemies, but Orlanth always told him to stay in the longhouse. Humakt worried about this. He had given his word to Orlanth always to obey his commands, but he had also given his word to Orlanth always to defend his tula. Now his two oaths were in conflict. Humakt thought long and hard about this, and then decided what to do. Soon, the winds started to blow hot and cold and fierce. It was an omen that the bad members of the Storm Tribe were coming to attack again. As soon as he felt the wind, Humakt left Orlanth’s hall. That way, when Orlanth gave the orders not to attack Valind, Humakt wasn’t there to hear them. Humakt was free to carry out his oath to defend Orlanth’s tula and, when Valind stepped onto the tula, Humakt stepped up to him and killed him. === Ernalda Heals the Tribe === Even after Ernalda told the Orlanth to make the Storm Tribe, there were problems. Even after Ernalda told Orlanth what laws to make, there were problems. Even after Ernalda told Orlanth how to judge the lawsuits, there were problems. Ernalda sat and thought about the problems and what she could do about them. She discovered that the problems were always worse just after Issaries had visited, so the next time he came, she watched very carefully. She saw that Issaries had many beautiful and wonderful things he had found in foreign lands. She saw that everyone in the Tribe wanted all the things, but Issaries wouldn’t just give them away. No, Issaries traded for them, so that everyone had to give something up to get something new. This made a lot of people unhappy, because they weren't any better off after Issaries had been than they were before, and now they were angry that there were lots of things that Issaries had that they didn’t, and that some people had more things than others. Ernalda knew what to do. One night, when everyone was sleeping, she took away all the things that Issaries had brought and hid them in her secret place. The next morning when everyone woke up, they couldn't find the things Issaries had brought and they were all very angry. They all came to Orlanth’s hall and demanded that he did something about it. Then Ernalda spoke to them and showed them that they still had the most important thing of all, and that was the Storm Tribe. Everyone realised that Ernalda was speaking the truth, and they were all happy again. Then Ernalda took everything out of her secret place, and everyone agreed to share them among themselves. == Characters == [[Peace Process characters|Characters for this episode]] are on a separate page. If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. [[Category:HeroQuest|Peace Process]] [[Category:Glorantha|Peace Process]] [[Category:Scenarios|Peace Process]] 62fc5df6bd749299369d478a3794b3dde61e1436 1884 1883 2009-01-08T21:53:46Z Neil 2 /* Scene II (Midpoint): When Impests Attack! */ Added picture wikitext text/x-wiki <blockquote>A [[:Category:HeroQuest|HeroQuest]] Episode by Neil Smith</blockquote> == Introduction == This is an episode for [[:Category:HeroQuest|HeroQuest]]. It was written to be played by five or six players (four at a pinch) in a single three to four hour session, typical of convention games. Six [[Peace Process characters|pre-generated characters]] are provided. It is designed as a drama, rather than a thriller: the engine of the scenario lies in the characters, their motivations, and the conflicts between them. The focus should be not so much on the events that unfold during the scenario as on how the heroes react and interact as a result of those events. I advise you to understand the inter-hero tensions fully before you run the scenario. Also, when running this episode, ensure that you give plenty of time for the players to chew the scenery and explore their heroes. This episode is set in an anonymous valley in the Quivin Mountains in Sartar. All the heroes are typical Orlanthi from two neighbouring clans as they attempt to bring an end to the feud that has damaged them for decades. This episode requires the '''HeroQuest''' core rulebook. '''Thunder Rebels''' and '''Storm Tribe''' are both extremely useful for background information. The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll probably need a copy of this myth as a handout for play. I'm told that '''Barbarian Adventures''' may come in handy, but I don't actually own it! {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Many contests are described in a short form, as '''''task''' (resistance): typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. |} == Backstory == {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Read this to the players at the start of the game. Don't mention that Sarotar and Gringle are conspiring against the other heroes. |} Until twenty years ago, the neighbouring Singing Rock and Hiord clans got on tolerably well; well enough for the young Singing Rock woman, '''Estave''', to marry '''Sarotar''', her true love, and leave her birth clan to join him in the Hiord clan. Soon after that, relations between the two clans began to break down and the usual cycle of cattle raids became more vicious on both sides. Eventually, a feud was declared and the fyrds skirmished. In that first battle, Brol the farmer was killed. '''Dorasa''', his wife, managed to have him resurrected but he had the Relife sickness. Brol turned to Humakt and became '''Mortality''', Champion of the Singing Rock clan. Since then, the feud has rumbled on, sometimes as a "cold war," sometimes erupting into violence. Recently, Estave and Dorasa have managed to broker a deal that will lead the two clans to peace. The two clans have agreed to end the feud and have charged Iristha (an Issarian from the Singing Rock clan) and Mikhil (a Lhankor Mhyite from the Hiord clan) to re-enact the "Issaries the Conciliator" myth. The quest will start in the Hirod tula tomorrow morning and finish atop the Singing Rocks tomorrow evening. The purpose of the quest is to discover the terms that will end the feud. When the heroquesters return, they will bring the resolution with them; both clans have promised to honour the terms on their return. The restoration of normal relations will be cemented by the marriage of '''Askos''' and Levru, a marriage arranged by Estave. Both clans have committed all their magical resources to this quest and, if either the quest or the marriage fail to happen tomorrow, the damage to both clans will be dreadful. The heroes have been charged with ensuring that the marriage takes place tomorrow evening, atop the Singing Rocks, when the heroquesters arrive. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |But all is not well. Sarotar and '''Gringle''', warleaders of the two opposing clans, have conspired to derail the quest by kidnapping Levru and ambushing the heroquesters on the Hero Plane. They each have their own reasons for this, but both agree that war is better than peace. |} == Summary == When the episode starts, all the heroes are at the edge of the Singing Rock tula, waiting for Levru to arrive. They are to form the honour guard for Levru as she travels to her new home there. Just as she arrives, her party is ambushed by a band of Gagarthi bandits (arranged by Sarotar), who kidnap Levru. The heroes will probably track the bandits, and may or may not discover their camp. They may be able to interrogate at least one Gagarthi and learn that they were hired to perform this ambush by someone in one of the clans. The heroes may try other ways of discovering more about the Gagarthi, but to little avail. The next day, during the Windsday rituals at the Singing Rocks, one of the clan's new weaponthanes is driven wild by impests and yavorlings and falls to his death from the Rocks. The heroes should soon discover that other weaponthanes are implicated in the plot to kidnap Levru. The heroes race to the weaponthanes' training ground where they discover Levru (unharmed) and a portal to the Hero Plane. The heroes step through the portal and foil an attempt by the conspirators to ambush and kill the heroquesters Iristha and Mikhil. At some point in all this, Gringle and Sarotar reveal themselves as the plot's ringleaders, probably as they attack the other heroes. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The players are almost certainly going to move a long way from the events described here. Let them. However, the characters carry the conflict with them (the important one is between Gringle & Sarotar and the rest of the heroes), and there is a tight deadline to work to, so they should bring themselves back on track. So long as you keep that in mind, you should be able to improvise interesting events around the heroes, whatever the players have them do. As examples, in one playtest, Sarotar confessed all just after the death at the Singing Rock, shopped Gringle (an NPC in that game) and asked for a chance to put right his mistakes: a very Orlanthi thing to do. In another session, the climax was when Mortality came to fulfil his vow to kill the defeated Sarotar, only for Estave to throw herself over her husband's body: Mortality killed them both. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Don't worry too much about the pacing in this episode. The heroes are likely to move through most of it quite quickly, but you're unlikely to finish too soon. The final scene, where Gringle , Sarotar, and the other conspirators stab the other heroes in the back, will take quite a while to play out. |} == The Episode == === Act I: Introduction === ==== Scene I: The Gagarthi Raid ==== ''Key Points: The Heroes get to strut their stuff, and some Gagarthi kidnap Levru.'' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |This scene is more about giving the players a chance to give their new characters a work out than influencing the plot. |} [[Image:Peace-process-map.png|right|thumb|200px|A map of the Singing Rock and Hiord tulas. This is an approximate map of the various locations of significance in this episode. The two tulas are in neighbouring valleys in the Quivin mountains. The land between them is mountainous and heavily forested, suitable only for hunting. The nearest the two tulas come to each other is at a pass between the high sheep pastures. ]] The scene opens with the heroes standing on the edge of the Singing Rock tula, waiting for Levru to arrive. They have all just left the Singing Rock stead where they have been preparing for Levru's arrival. Mortality has witnessed the Orlanthi Greeting that gave Sarotar the Blanket gift and Estave the Meat gift; this means that the Singing Rock clan has pledged hospitality and safety to both of them ('''Thunder Rebels''', pp. 34-36). The heroes are to be Levru's honour guard as she crosses the Singing Rock tula. She is being brought across the Hiord tula by an honour guard of Hiord weaponthanes. Just as the Hiord party comes into earshot, the heroes can use their (magical) perception abilities to '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. If they succeed, describe the feel of a hot, dry wind, the sand stinging their cheek, and the smell of hot flint. As soon as they think they have been rumbled, or when the Hiord party turn up, the Gagarthi attack. Many of them attack the Hiord party; the remainder try to hold back the heroes. The heroes face one Gagarthi leader per hero and two henchmen per hero , together with the odd whirlvish or two. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Gagarthi Leader: Sword & Shield Fighting 14W, (+3 auto-augment from Tough Outlaw = 5W), Walk on Winds 6W, Stinging Winds 2W, Tough Outlaw 5W, 39AP + 27AP + 27AP (from two henchmen) = 93AP. |- |Gagarthi Henchman: Sword & Shield Fighting 5W, Stinging Winds 18, Tough Outlaw 19; treat as followers, providing 27AP each. |- |Whirlvish: Abrade 10W, Spirit combat 15, 32AP. |} If the heroes win this encounter, they drive their assailants back into the Hiord party, who then capture and escape with Levru. If the heroes lose, they are forced back and fail to prevent the Gagarthi from kidnapping Levru. Try to ensure that none of the Gagarthi leaders are captured. ==== Scene II (Plot Turn I): Kidnap and Recriminations ==== ''Key Points: The heroes discover that Levru has been taken, and that the peace deal is not a foregone conclusion.'' When the Gagarthi leave, the heroes will want to investigate the Hiord party. They will discover that the 12 Hiordlings have been defeated, many of them are injured, and they have just noticed that Levru has been taken, snatched by a wind-walking Gagarthi. Many of the Hiordlings will blame the heroes for the attack, claiming that they were on Singing Rock lands and that the Singing Rocks are responsible for the injuries. Someone might even suggest that they wanted the Gagarthi to attack (though they have no evidence to support this: it's just hot-headed talking). Tempers can be calmed in a number of ways: tending to the wounded, boasting of how the Gagarthi were defeated, vows to avenge the terrible crime. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |As the heroes take stock of the situation, reinforce that the marriage between Levru and Askos is an essential part of the peace deal. If the marriage doesn't occur at the prescribed place and time, the feud will continue at great cost to both clans. |} === Act II: Conflict === ==== Scene I: Hunting the Wild Hunters ==== ''Key Points: The heroes discover that the Gagarthi had an informer from one of the two clans.'' The ball is now in the heroes' court. They are likely to want to gain some information about the Gagarthi and Levru: where they are, why they attacked, how they knew, and what are they planning for Levru. There are a few avenues that could be open to them. There could be Gagarthi prisoners to interrogate. Some Gagarthi fled on foot and can be tracked. The god-talker heroes may want to use divinations. Also, someone will have to tell the Singing Rock clan the bad news of Levru's kidnap. * ''Interrogating Gagarthi'': The bandits are fanatics, and all declared outlaw. They know that if they are ever brought to justice they are almost certain to be executed, so they have no real reason to co-operate. This makes interrogation difficult. The heroes know this (inform the players if need by), but they are likely anyway to want to interrogate them anyway. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate Gagarthi Bandits''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Resistance'': 12W2 |- |''Complete or Major Defeat'': The bandit either commits suicide or manages to snatch a hostage. |- |''Other Defeat or Tie'': The bandit reveals nothing. |- |''Marginal Victory'': The bandit hints that they knew that someone would be here. |- |''Minor Victory'': The bandits were waiting specifically for this group of Hiord clansfolk. |- |''Major Victory'': The Gagarthi were hired to kidnap “the girl.” |- |''Complete Victory'': This was definitely an inside job, connected to some big ritual. He doesn't know the identity of the insider. |} Heroes may '''''Recognise the Bandit''' (15W): Relationship to Clan'' and recall the bandit's name and deeds, and reduce the bandit's resistance according to the result of that contest. * ''Tracking the Gagarthi'': The wind-walking Gagarthi leaders are the ones who stole Levru, and they are difficult to track. However, the other Gagarthi who escaped are easier to follow: '''''Track the Gagarthi''' (3W): Track, Scout (-5), Singing Rock Geography (-10), Hiord Geography (-10)''. Any victory allows the heroes to follow the trail. It soon gets easier to follow once the wind-walking bandits come to earth. After a few hours of hard walking over some increasingly rough terrain (up and down some some very steep and heavily wooded slopes), the heroes come to the Gagarthi's (deserted) camp. The camp has been packed up recently and the firepit it still hot. Observant heroes will notice two fresh trails leaving the camp, one more heavily used than the other. Both peter out after a mile or so as they pass over bare, rocky ground in the fading light. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Their job done, the Gagarthi have been paid off and have left the area. Levru has been taken by some weaponthanes to the training camp in the Singing Rock tula. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |One potential problem point is persuading the heroes that they've lost the Gagarthi's trail. Stress the difficulty of following the trail, the worsening conditions, and time pressure the heroes are under to force them into trying another tack. |} * ''Asking the gods'': The heroes may ask the gods, or at least the Singing Rock clan wyter, where Levru and the Gagarthi are. (The Singing Rocks are the best bet, as most of the heroes have magical connections there they can call on.) Let the heroes go to town in describing the preparations, and involve the heroes in persuading the clan to support them and provide community support and objects to sacrifice. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Ask the Gods about Levru and the Gagarthi''' |- |''Appropriate abilities'': Pray to Orlanth, Pray to Ernalda, Devotee of <Deity> (-5) |- |''Modifiers and Augments'': Length of ritual (+3/hr), <Deity> Myths, various Relationships and Personality traits (for emotional involvement), various Relationships (for community support), auspicious location (e.g. the Singing Rocks) (+10), value of sacrifices (up to +10), not a member of the Singing Rock clan (-20), penalties for fatigue (contest based on time and exertion up to now). See also the Ritual Magic section, HeroQuest pp. 101-2. |- |''Resistance'': 10W3 |- |''Complete, Major, or Minor Defeat'': The heroes fail to learn anything. |- |''Marginal Defeat or Tie'': The Singing Rock wyter tells of a disturbance in the clan. |- |''Marginal Victory'': There is something wrong in the clan. Neither Levru nor the Gagarthi are touched by the wind or the earth within the Singing Rock tula. |- |''Minor Victory'': There is something wrong in the clan. Levru is within the tula; the Gagarthi are not. |- |''Major Victory'': There is discord in the Song of the Rocks, discord in the very heart of the tula. Levru is hidden within the tula. |- |''Complete Victory'': Kinstrife! Kinstrife! Kinstrife brought on by people both here and elsewhere! A vision of Levru being bound by a man with Singing Rock clan tatoos. |} By the time the rituals are complete, the heroes are liable to be too fatigued to do much more. They should go to bed disheartened that the peace deal is in danger of failure. ==== Scene II (Midpoint): When Impests Attack! ==== ''Key Points: The conspiracy is revealed when a weaponthane is driven to his death by Impests.'' [[Image:Peace-process-singing-rocks.png|right|thumb|400px|The Singing Rocks are the centre of the clan's lands. From the east, the ground rises up to a tall and increasingly narrow hill, rather like Castle Mount in Edinburgh. At the western end, the soil peters out to expose a jagged, rocky column over 150' high. As the wind blows around the hill, it swirls between the column and the cliff and sings and moans in honour of Orlanth and Ernalda. This swirling wind is the clan's wyter, and its song is understood by the chieftain. The peak of this hill is the holiest place in the tula, as it symbolises the marriage of Earth and Air; most rituals are led from here. A minor test of faith is to leap from the hill to the column; a major one is to leap from the column into the air, trusting that Orlanth will suspend you.]] At dawn the next morning, the heroes join the assembling clan atop the Singing Rocks. The mood is sombre as the clan contemplate the impending failure of the heroquest. The heroes may contemplate trying to persuade Sarostip that the quest should be abandoned, but they know that the consequences abandoning it would be worse than the consequences of failure, assuming that they could get word to Iristha nd Mikhil in time. The heroes may '''''notice that some weaponthanes are missing''' (2W): Relationship to warband, Relationship to clan (-8)'', though they won't necessarily know what that means. As the early morning worship starts, there is a scream from in the congregation as one of the weaponthanes starts to flail around wildly. Characters may '''''sense the clustering agents of reprisal''' (14): Sense gods nearby, Piety (-5)'' as the man is attacked by yavorlings and harulings (sparkers and impests) ('''Thunder Rebels''', pp. 81-82). As he attempts to flee, he runs blindly off the edge of the cliff and falls to smash on the rocks far below. Everyone is shocked, both because of the sudden and violent death, and for the calamitous omen it represents. The agents of reprisal follow him down and circle his corpse. They then rise again to swirl around the huddled clan. Perhaps their work is not yet done... ==== Scene III: The Conspiracy Uncovered ==== ''Key Points: The heroes discover that the conspiracy is led by the weaponthanes.'' All the Singing Rock heroes will recognise the dead man as Willem, a new weaponthane. They are likely to want to have pointed words with his colleagues, particularly in light of their suspicions that there is a traitor in the clan. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate the Weaponthanes''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Modifiers and Augments'': Threats of violence (-20 to +20, depending on contest v. resistance of 15W), promises of amnesty (as threats of violence), Relationship to warriors. |- |''Resistance'': 5W2 |- |''Complete or Major Defeat'': The militia manage to place the blame on the few Hiord clansmen present and incite a lynch mob. |- |''Other Defeat or Tie'': The weaponthanes reveal nothing. |- |''Marginal Victory'': Someone says that the weaponthanes have been plotting something, but there are no more details. |- |''Minor Victory'': A warrior with a mouthful of dirt blurts out that “They forced me to keep quiet!” and reveals that some weaponthanes (inlcuding Gringle, if he's an NPC) want the feud to continue. |- |''Major Victory'': The weaponthanes reveal that some of them have decided the feud should continue, and are conspiring to ensure that it does. |- |''Complete Victory'': The weaponthanes tell everything, including the involvement of Sarotar and Gringle, then attack them! |} ==== Scene IV (Plot Turn II): To the Training Ground ==== ''Key Points: The heroes rescue Levru and enter the Hero Plane.'' The heroes will either have evidence that the weaponthanes are involved, or suspect that this is the case. In either case, they are likely to want to pay a visit to their training ground, a mile or so from the stead. The training ground is a small clearing, about a hundred yards across, on a level piece of ground in the hills above the Singing Rocks. Around the field are archery butts, straw-stuffed dummies for sword practice, and other such paraphernalia. When the heroes arrive, the field is deserted. One one side of the field is the weaponthanes' longhouse. It is silent and dark, though smoke is rising from the smoke hole. The only doors are in the middle of the long side facing the field; they are closed and barred from the inside. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Play up the paranoia while the heroes are moving into the longhouse. They shouldn't know the place is deserted until they rush the longhouse. |} When the heroes enter the longhouse, three things are immediately apparent. First is the bullock in the middle of the floor, lying in a pool if its own blood. Second is Levru, gagged and tied to a chair at one end of the longhouse. Third is a large portal to the Hero Plane at the other end of the longhouse. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The portal looks rather like the Stargate from the TV series, if that would help your players visualise it. |} The bullock has been ritually slaughtered as part of a ceremony. Any god-talker will be able to deduce that the ritual was probably the one to open the portal. Levru is unharmed and will tell the heroes that, about an hour ago, a group of four weaponthanes opened the portal. They were talking about ambushing Iristha and Mikhil on the Hero Plane. Levru will tell the heroes that she recognised two of the weaponthanes as being Hiord clansmen; the remainder had Singing Rock clan tattoos. The heroes should realise that they are the only ones who can save the peace treaty, by eliminating the weaponthanes so that Iristha and Mikhil can complete their hero quest. The heroes will have to enter the hero plane. === Act III: Resolution === ==== Scene I: Rumble on the Hero Plane ==== ''Key Points: The heroes discover the full extent of the conspiracy, and it is either defeated or it isn't.'' When the heroes enter the hero plane, they can easily track the weaponthanes through a small patch of forest to where they are lying hidden above a trail, waiting to ambush Iristha and Mikhil when they pass through into the Big Teeths' lands. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Emphasise the hyper-realistic nature of the hero plane. Colours are more vibrant; vision is more acute; smells are more intense; sounds are more moving. |} The conspirators, Baranwulf, Illig, Kagrad, and Verlain (Ranged attack 16W, Close attack 6W2, Resist Magic 14W, Hardened Consiprator 15W, all after augments) will attack the heroes if attacked themselves, or if they are being beaten in a battle of wits and words. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The hero plane is the world of myth, and you should bring this out in your desciptions. When the conspirators attack, they are taking the part of the Long Teeth; their leather armour regrows stiff, spotted fur; it's difficult to hear words in their animalistic snarling; in the confusion of melee, are they attacking with their swords or their foot-long canines? Heroes, particularly Dorasa and Estave, trumpet and bellow and crush their enemies beneath their feet. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The normal physical rules don't entirely apply on the hero plane. If a character explicitly invokes a myth to justify their actions, give them a bonus of +5 or so. If they continue to invoke the same myth successfully, increase the bonus or allow them to decrease their opponents' bonus. The same bonus applies to everyone else who invokes the same myth. Some mythlets are in the appendix; the conspirators are invoking the “Big Teeth kill the Intruders” myth. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Just as Earth defeats Storm in the elemental progression, so Earth cultists have an advantage over Storm cultists in the hero plane. Give Dorasa and Estave a bonus of +5 to resist all actions by an Orlanth cultist, rising to +10 when resisting magic with magic (alternatively, use the rules from Masters of Luck and Death, p. 52) |} At some point in this confrontation, if not before, the players of Gringle and Sarotar will want to reveal their true objectives and stab the other heroes in the back. Let them. The whole episode has been building up to this struggle between the player heroes, and its is their actions that will determine whether the peace process succeeds or not. ==== Scene II: Denouement ==== If the conspirators are victorious, they move on to ambush and kill Iristha and Mikhil. Normal life in the two clans is shattered as open, total war breaks out between them. The weaponthanes gain power and prestige, but at a dreadful cost. If the heroes are successful, they hear the sounds of Issaries and Lhankor Mhy passing below just as the conspirators are defeated. The heroes can return to the Singing Rocks in time for the wedding between Askos and Levru. If the heroes succeed, but Askos cannot or will not marry Levru, another bachelor can be found but the resulting peace treaty will be neither as strong nor as long-lasting. == Acknowledgements == I'd like to thank Rob Davies for his feedback from running and playtesting this scenario. I'd also like to thank Tim Ellis, Newt Newport, and Paul Kay for their insightful reviews and comments. == Appendix: Mythlets == The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll need a copy of this myth as a handout for play. === Orlanth Banishes The Thieves === One day, Orlanth came home from a day's work in the fields to find that his favourite neck torc was missing. He looked all over his stead for it: he lifted up the mattresses, he flew up into the rafters, he swirled into all the corners of the longhouse, but he couldn't find his neck torc anywhere. Later, at dinner, he heard Vadrus, his brother, boasting to his friends in the corner of the longhouse. Orlanth left his dinner to see what Vadrus was going on about this time. When he got there, he saw that Vadrus was wearing Orlanth's favourite neck torc! "That’s mine!" shouted Orlanth. "Give it back." "No," said Vadrus. "I ... found it, in the field. It’s mine now." "But my laws say that even when something is lost, it still belongs to its owner. It’s mine." "I claim it by right of conquest!" bellowed Vadrus. "If you want it, come an take it!" But instead of reaching for his sword, Orlanth reached for his lawstaff. "This is my law," he said, "and you will obey it!" And with that he cracked Vadrus's head with the staff and Vadrus dropped the torc. Vadrus and his friends ran out of the longhouse. Orlanth chased them all the way to the edge of his tula and told them never to come back. === Humakt Slays the Enemies === After Humakt had reclaimed Death, and after Orlanth had given him a place in his hall, Humakt faced many enemies. Whenever the Sky Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Night Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Plant Tribe attacked, Humakt killed their warriors and drove them back. But when others in the Storm Tribe attacked, no-one would fight them. Even thought the Storm Tribe were enemies, they were still kin and Orlanth would not attack them. Vadrus, and Valind, and even Ragnaglar came to Orlanth's tula to fight and steal, but Orlanth would not risk kinstrife by driving them off. Humakt said that he had no kin, and so could attack the enemies, but Orlanth always told him to stay in the longhouse. Humakt worried about this. He had given his word to Orlanth always to obey his commands, but he had also given his word to Orlanth always to defend his tula. Now his two oaths were in conflict. Humakt thought long and hard about this, and then decided what to do. Soon, the winds started to blow hot and cold and fierce. It was an omen that the bad members of the Storm Tribe were coming to attack again. As soon as he felt the wind, Humakt left Orlanth’s hall. That way, when Orlanth gave the orders not to attack Valind, Humakt wasn’t there to hear them. Humakt was free to carry out his oath to defend Orlanth’s tula and, when Valind stepped onto the tula, Humakt stepped up to him and killed him. === Ernalda Heals the Tribe === Even after Ernalda told the Orlanth to make the Storm Tribe, there were problems. Even after Ernalda told Orlanth what laws to make, there were problems. Even after Ernalda told Orlanth how to judge the lawsuits, there were problems. Ernalda sat and thought about the problems and what she could do about them. She discovered that the problems were always worse just after Issaries had visited, so the next time he came, she watched very carefully. She saw that Issaries had many beautiful and wonderful things he had found in foreign lands. She saw that everyone in the Tribe wanted all the things, but Issaries wouldn’t just give them away. No, Issaries traded for them, so that everyone had to give something up to get something new. This made a lot of people unhappy, because they weren't any better off after Issaries had been than they were before, and now they were angry that there were lots of things that Issaries had that they didn’t, and that some people had more things than others. Ernalda knew what to do. One night, when everyone was sleeping, she took away all the things that Issaries had brought and hid them in her secret place. The next morning when everyone woke up, they couldn't find the things Issaries had brought and they were all very angry. They all came to Orlanth’s hall and demanded that he did something about it. Then Ernalda spoke to them and showed them that they still had the most important thing of all, and that was the Storm Tribe. Everyone realised that Ernalda was speaking the truth, and they were all happy again. Then Ernalda took everything out of her secret place, and everyone agreed to share them among themselves. == Characters == [[Peace Process characters|Characters for this episode]] are on a separate page. If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. [[Category:HeroQuest|Peace Process]] [[Category:Glorantha|Peace Process]] [[Category:Scenarios|Peace Process]] 461009fccb9584b619584c5231d35fb4c3895599 1885 1884 2009-01-08T21:54:20Z Neil 2 /* Scene I: The Gagarthi Raid */ Tweaked size of map wikitext text/x-wiki <blockquote>A [[:Category:HeroQuest|HeroQuest]] Episode by Neil Smith</blockquote> == Introduction == This is an episode for [[:Category:HeroQuest|HeroQuest]]. It was written to be played by five or six players (four at a pinch) in a single three to four hour session, typical of convention games. Six [[Peace Process characters|pre-generated characters]] are provided. It is designed as a drama, rather than a thriller: the engine of the scenario lies in the characters, their motivations, and the conflicts between them. The focus should be not so much on the events that unfold during the scenario as on how the heroes react and interact as a result of those events. I advise you to understand the inter-hero tensions fully before you run the scenario. Also, when running this episode, ensure that you give plenty of time for the players to chew the scenery and explore their heroes. This episode is set in an anonymous valley in the Quivin Mountains in Sartar. All the heroes are typical Orlanthi from two neighbouring clans as they attempt to bring an end to the feud that has damaged them for decades. This episode requires the '''HeroQuest''' core rulebook. '''Thunder Rebels''' and '''Storm Tribe''' are both extremely useful for background information. The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll probably need a copy of this myth as a handout for play. I'm told that '''Barbarian Adventures''' may come in handy, but I don't actually own it! {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Many contests are described in a short form, as '''''task''' (resistance): typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. |} == Backstory == {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Read this to the players at the start of the game. Don't mention that Sarotar and Gringle are conspiring against the other heroes. |} Until twenty years ago, the neighbouring Singing Rock and Hiord clans got on tolerably well; well enough for the young Singing Rock woman, '''Estave''', to marry '''Sarotar''', her true love, and leave her birth clan to join him in the Hiord clan. Soon after that, relations between the two clans began to break down and the usual cycle of cattle raids became more vicious on both sides. Eventually, a feud was declared and the fyrds skirmished. In that first battle, Brol the farmer was killed. '''Dorasa''', his wife, managed to have him resurrected but he had the Relife sickness. Brol turned to Humakt and became '''Mortality''', Champion of the Singing Rock clan. Since then, the feud has rumbled on, sometimes as a "cold war," sometimes erupting into violence. Recently, Estave and Dorasa have managed to broker a deal that will lead the two clans to peace. The two clans have agreed to end the feud and have charged Iristha (an Issarian from the Singing Rock clan) and Mikhil (a Lhankor Mhyite from the Hiord clan) to re-enact the "Issaries the Conciliator" myth. The quest will start in the Hirod tula tomorrow morning and finish atop the Singing Rocks tomorrow evening. The purpose of the quest is to discover the terms that will end the feud. When the heroquesters return, they will bring the resolution with them; both clans have promised to honour the terms on their return. The restoration of normal relations will be cemented by the marriage of '''Askos''' and Levru, a marriage arranged by Estave. Both clans have committed all their magical resources to this quest and, if either the quest or the marriage fail to happen tomorrow, the damage to both clans will be dreadful. The heroes have been charged with ensuring that the marriage takes place tomorrow evening, atop the Singing Rocks, when the heroquesters arrive. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |But all is not well. Sarotar and '''Gringle''', warleaders of the two opposing clans, have conspired to derail the quest by kidnapping Levru and ambushing the heroquesters on the Hero Plane. They each have their own reasons for this, but both agree that war is better than peace. |} == Summary == When the episode starts, all the heroes are at the edge of the Singing Rock tula, waiting for Levru to arrive. They are to form the honour guard for Levru as she travels to her new home there. Just as she arrives, her party is ambushed by a band of Gagarthi bandits (arranged by Sarotar), who kidnap Levru. The heroes will probably track the bandits, and may or may not discover their camp. They may be able to interrogate at least one Gagarthi and learn that they were hired to perform this ambush by someone in one of the clans. The heroes may try other ways of discovering more about the Gagarthi, but to little avail. The next day, during the Windsday rituals at the Singing Rocks, one of the clan's new weaponthanes is driven wild by impests and yavorlings and falls to his death from the Rocks. The heroes should soon discover that other weaponthanes are implicated in the plot to kidnap Levru. The heroes race to the weaponthanes' training ground where they discover Levru (unharmed) and a portal to the Hero Plane. The heroes step through the portal and foil an attempt by the conspirators to ambush and kill the heroquesters Iristha and Mikhil. At some point in all this, Gringle and Sarotar reveal themselves as the plot's ringleaders, probably as they attack the other heroes. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The players are almost certainly going to move a long way from the events described here. Let them. However, the characters carry the conflict with them (the important one is between Gringle & Sarotar and the rest of the heroes), and there is a tight deadline to work to, so they should bring themselves back on track. So long as you keep that in mind, you should be able to improvise interesting events around the heroes, whatever the players have them do. As examples, in one playtest, Sarotar confessed all just after the death at the Singing Rock, shopped Gringle (an NPC in that game) and asked for a chance to put right his mistakes: a very Orlanthi thing to do. In another session, the climax was when Mortality came to fulfil his vow to kill the defeated Sarotar, only for Estave to throw herself over her husband's body: Mortality killed them both. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Don't worry too much about the pacing in this episode. The heroes are likely to move through most of it quite quickly, but you're unlikely to finish too soon. The final scene, where Gringle , Sarotar, and the other conspirators stab the other heroes in the back, will take quite a while to play out. |} == The Episode == === Act I: Introduction === ==== Scene I: The Gagarthi Raid ==== ''Key Points: The Heroes get to strut their stuff, and some Gagarthi kidnap Levru.'' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |This scene is more about giving the players a chance to give their new characters a work out than influencing the plot. |} [[Image:Peace-process-map.png|right|thumb|400px|A map of the Singing Rock and Hiord tulas. This is an approximate map of the various locations of significance in this episode. The two tulas are in neighbouring valleys in the Quivin mountains. The land between them is mountainous and heavily forested, suitable only for hunting. The nearest the two tulas come to each other is at a pass between the high sheep pastures. ]] The scene opens with the heroes standing on the edge of the Singing Rock tula, waiting for Levru to arrive. They have all just left the Singing Rock stead where they have been preparing for Levru's arrival. Mortality has witnessed the Orlanthi Greeting that gave Sarotar the Blanket gift and Estave the Meat gift; this means that the Singing Rock clan has pledged hospitality and safety to both of them ('''Thunder Rebels''', pp. 34-36). The heroes are to be Levru's honour guard as she crosses the Singing Rock tula. She is being brought across the Hiord tula by an honour guard of Hiord weaponthanes. Just as the Hiord party comes into earshot, the heroes can use their (magical) perception abilities to '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. If they succeed, describe the feel of a hot, dry wind, the sand stinging their cheek, and the smell of hot flint. As soon as they think they have been rumbled, or when the Hiord party turn up, the Gagarthi attack. Many of them attack the Hiord party; the remainder try to hold back the heroes. The heroes face one Gagarthi leader per hero and two henchmen per hero , together with the odd whirlvish or two. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Gagarthi Leader: Sword & Shield Fighting 14W, (+3 auto-augment from Tough Outlaw = 5W), Walk on Winds 6W, Stinging Winds 2W, Tough Outlaw 5W, 39AP + 27AP + 27AP (from two henchmen) = 93AP. |- |Gagarthi Henchman: Sword & Shield Fighting 5W, Stinging Winds 18, Tough Outlaw 19; treat as followers, providing 27AP each. |- |Whirlvish: Abrade 10W, Spirit combat 15, 32AP. |} If the heroes win this encounter, they drive their assailants back into the Hiord party, who then capture and escape with Levru. If the heroes lose, they are forced back and fail to prevent the Gagarthi from kidnapping Levru. Try to ensure that none of the Gagarthi leaders are captured. ==== Scene II (Plot Turn I): Kidnap and Recriminations ==== ''Key Points: The heroes discover that Levru has been taken, and that the peace deal is not a foregone conclusion.'' When the Gagarthi leave, the heroes will want to investigate the Hiord party. They will discover that the 12 Hiordlings have been defeated, many of them are injured, and they have just noticed that Levru has been taken, snatched by a wind-walking Gagarthi. Many of the Hiordlings will blame the heroes for the attack, claiming that they were on Singing Rock lands and that the Singing Rocks are responsible for the injuries. Someone might even suggest that they wanted the Gagarthi to attack (though they have no evidence to support this: it's just hot-headed talking). Tempers can be calmed in a number of ways: tending to the wounded, boasting of how the Gagarthi were defeated, vows to avenge the terrible crime. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |As the heroes take stock of the situation, reinforce that the marriage between Levru and Askos is an essential part of the peace deal. If the marriage doesn't occur at the prescribed place and time, the feud will continue at great cost to both clans. |} === Act II: Conflict === ==== Scene I: Hunting the Wild Hunters ==== ''Key Points: The heroes discover that the Gagarthi had an informer from one of the two clans.'' The ball is now in the heroes' court. They are likely to want to gain some information about the Gagarthi and Levru: where they are, why they attacked, how they knew, and what are they planning for Levru. There are a few avenues that could be open to them. There could be Gagarthi prisoners to interrogate. Some Gagarthi fled on foot and can be tracked. The god-talker heroes may want to use divinations. Also, someone will have to tell the Singing Rock clan the bad news of Levru's kidnap. * ''Interrogating Gagarthi'': The bandits are fanatics, and all declared outlaw. They know that if they are ever brought to justice they are almost certain to be executed, so they have no real reason to co-operate. This makes interrogation difficult. The heroes know this (inform the players if need by), but they are likely anyway to want to interrogate them anyway. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate Gagarthi Bandits''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Resistance'': 12W2 |- |''Complete or Major Defeat'': The bandit either commits suicide or manages to snatch a hostage. |- |''Other Defeat or Tie'': The bandit reveals nothing. |- |''Marginal Victory'': The bandit hints that they knew that someone would be here. |- |''Minor Victory'': The bandits were waiting specifically for this group of Hiord clansfolk. |- |''Major Victory'': The Gagarthi were hired to kidnap “the girl.” |- |''Complete Victory'': This was definitely an inside job, connected to some big ritual. He doesn't know the identity of the insider. |} Heroes may '''''Recognise the Bandit''' (15W): Relationship to Clan'' and recall the bandit's name and deeds, and reduce the bandit's resistance according to the result of that contest. * ''Tracking the Gagarthi'': The wind-walking Gagarthi leaders are the ones who stole Levru, and they are difficult to track. However, the other Gagarthi who escaped are easier to follow: '''''Track the Gagarthi''' (3W): Track, Scout (-5), Singing Rock Geography (-10), Hiord Geography (-10)''. Any victory allows the heroes to follow the trail. It soon gets easier to follow once the wind-walking bandits come to earth. After a few hours of hard walking over some increasingly rough terrain (up and down some some very steep and heavily wooded slopes), the heroes come to the Gagarthi's (deserted) camp. The camp has been packed up recently and the firepit it still hot. Observant heroes will notice two fresh trails leaving the camp, one more heavily used than the other. Both peter out after a mile or so as they pass over bare, rocky ground in the fading light. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Their job done, the Gagarthi have been paid off and have left the area. Levru has been taken by some weaponthanes to the training camp in the Singing Rock tula. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |One potential problem point is persuading the heroes that they've lost the Gagarthi's trail. Stress the difficulty of following the trail, the worsening conditions, and time pressure the heroes are under to force them into trying another tack. |} * ''Asking the gods'': The heroes may ask the gods, or at least the Singing Rock clan wyter, where Levru and the Gagarthi are. (The Singing Rocks are the best bet, as most of the heroes have magical connections there they can call on.) Let the heroes go to town in describing the preparations, and involve the heroes in persuading the clan to support them and provide community support and objects to sacrifice. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Ask the Gods about Levru and the Gagarthi''' |- |''Appropriate abilities'': Pray to Orlanth, Pray to Ernalda, Devotee of <Deity> (-5) |- |''Modifiers and Augments'': Length of ritual (+3/hr), <Deity> Myths, various Relationships and Personality traits (for emotional involvement), various Relationships (for community support), auspicious location (e.g. the Singing Rocks) (+10), value of sacrifices (up to +10), not a member of the Singing Rock clan (-20), penalties for fatigue (contest based on time and exertion up to now). See also the Ritual Magic section, HeroQuest pp. 101-2. |- |''Resistance'': 10W3 |- |''Complete, Major, or Minor Defeat'': The heroes fail to learn anything. |- |''Marginal Defeat or Tie'': The Singing Rock wyter tells of a disturbance in the clan. |- |''Marginal Victory'': There is something wrong in the clan. Neither Levru nor the Gagarthi are touched by the wind or the earth within the Singing Rock tula. |- |''Minor Victory'': There is something wrong in the clan. Levru is within the tula; the Gagarthi are not. |- |''Major Victory'': There is discord in the Song of the Rocks, discord in the very heart of the tula. Levru is hidden within the tula. |- |''Complete Victory'': Kinstrife! Kinstrife! Kinstrife brought on by people both here and elsewhere! A vision of Levru being bound by a man with Singing Rock clan tatoos. |} By the time the rituals are complete, the heroes are liable to be too fatigued to do much more. They should go to bed disheartened that the peace deal is in danger of failure. ==== Scene II (Midpoint): When Impests Attack! ==== ''Key Points: The conspiracy is revealed when a weaponthane is driven to his death by Impests.'' [[Image:Peace-process-singing-rocks.png|right|thumb|400px|The Singing Rocks are the centre of the clan's lands. From the east, the ground rises up to a tall and increasingly narrow hill, rather like Castle Mount in Edinburgh. At the western end, the soil peters out to expose a jagged, rocky column over 150' high. As the wind blows around the hill, it swirls between the column and the cliff and sings and moans in honour of Orlanth and Ernalda. This swirling wind is the clan's wyter, and its song is understood by the chieftain. The peak of this hill is the holiest place in the tula, as it symbolises the marriage of Earth and Air; most rituals are led from here. A minor test of faith is to leap from the hill to the column; a major one is to leap from the column into the air, trusting that Orlanth will suspend you.]] At dawn the next morning, the heroes join the assembling clan atop the Singing Rocks. The mood is sombre as the clan contemplate the impending failure of the heroquest. The heroes may contemplate trying to persuade Sarostip that the quest should be abandoned, but they know that the consequences abandoning it would be worse than the consequences of failure, assuming that they could get word to Iristha nd Mikhil in time. The heroes may '''''notice that some weaponthanes are missing''' (2W): Relationship to warband, Relationship to clan (-8)'', though they won't necessarily know what that means. As the early morning worship starts, there is a scream from in the congregation as one of the weaponthanes starts to flail around wildly. Characters may '''''sense the clustering agents of reprisal''' (14): Sense gods nearby, Piety (-5)'' as the man is attacked by yavorlings and harulings (sparkers and impests) ('''Thunder Rebels''', pp. 81-82). As he attempts to flee, he runs blindly off the edge of the cliff and falls to smash on the rocks far below. Everyone is shocked, both because of the sudden and violent death, and for the calamitous omen it represents. The agents of reprisal follow him down and circle his corpse. They then rise again to swirl around the huddled clan. Perhaps their work is not yet done... ==== Scene III: The Conspiracy Uncovered ==== ''Key Points: The heroes discover that the conspiracy is led by the weaponthanes.'' All the Singing Rock heroes will recognise the dead man as Willem, a new weaponthane. They are likely to want to have pointed words with his colleagues, particularly in light of their suspicions that there is a traitor in the clan. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate the Weaponthanes''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Modifiers and Augments'': Threats of violence (-20 to +20, depending on contest v. resistance of 15W), promises of amnesty (as threats of violence), Relationship to warriors. |- |''Resistance'': 5W2 |- |''Complete or Major Defeat'': The militia manage to place the blame on the few Hiord clansmen present and incite a lynch mob. |- |''Other Defeat or Tie'': The weaponthanes reveal nothing. |- |''Marginal Victory'': Someone says that the weaponthanes have been plotting something, but there are no more details. |- |''Minor Victory'': A warrior with a mouthful of dirt blurts out that “They forced me to keep quiet!” and reveals that some weaponthanes (inlcuding Gringle, if he's an NPC) want the feud to continue. |- |''Major Victory'': The weaponthanes reveal that some of them have decided the feud should continue, and are conspiring to ensure that it does. |- |''Complete Victory'': The weaponthanes tell everything, including the involvement of Sarotar and Gringle, then attack them! |} ==== Scene IV (Plot Turn II): To the Training Ground ==== ''Key Points: The heroes rescue Levru and enter the Hero Plane.'' The heroes will either have evidence that the weaponthanes are involved, or suspect that this is the case. In either case, they are likely to want to pay a visit to their training ground, a mile or so from the stead. The training ground is a small clearing, about a hundred yards across, on a level piece of ground in the hills above the Singing Rocks. Around the field are archery butts, straw-stuffed dummies for sword practice, and other such paraphernalia. When the heroes arrive, the field is deserted. One one side of the field is the weaponthanes' longhouse. It is silent and dark, though smoke is rising from the smoke hole. The only doors are in the middle of the long side facing the field; they are closed and barred from the inside. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Play up the paranoia while the heroes are moving into the longhouse. They shouldn't know the place is deserted until they rush the longhouse. |} When the heroes enter the longhouse, three things are immediately apparent. First is the bullock in the middle of the floor, lying in a pool if its own blood. Second is Levru, gagged and tied to a chair at one end of the longhouse. Third is a large portal to the Hero Plane at the other end of the longhouse. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The portal looks rather like the Stargate from the TV series, if that would help your players visualise it. |} The bullock has been ritually slaughtered as part of a ceremony. Any god-talker will be able to deduce that the ritual was probably the one to open the portal. Levru is unharmed and will tell the heroes that, about an hour ago, a group of four weaponthanes opened the portal. They were talking about ambushing Iristha and Mikhil on the Hero Plane. Levru will tell the heroes that she recognised two of the weaponthanes as being Hiord clansmen; the remainder had Singing Rock clan tattoos. The heroes should realise that they are the only ones who can save the peace treaty, by eliminating the weaponthanes so that Iristha and Mikhil can complete their hero quest. The heroes will have to enter the hero plane. === Act III: Resolution === ==== Scene I: Rumble on the Hero Plane ==== ''Key Points: The heroes discover the full extent of the conspiracy, and it is either defeated or it isn't.'' When the heroes enter the hero plane, they can easily track the weaponthanes through a small patch of forest to where they are lying hidden above a trail, waiting to ambush Iristha and Mikhil when they pass through into the Big Teeths' lands. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Emphasise the hyper-realistic nature of the hero plane. Colours are more vibrant; vision is more acute; smells are more intense; sounds are more moving. |} The conspirators, Baranwulf, Illig, Kagrad, and Verlain (Ranged attack 16W, Close attack 6W2, Resist Magic 14W, Hardened Consiprator 15W, all after augments) will attack the heroes if attacked themselves, or if they are being beaten in a battle of wits and words. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The hero plane is the world of myth, and you should bring this out in your desciptions. When the conspirators attack, they are taking the part of the Long Teeth; their leather armour regrows stiff, spotted fur; it's difficult to hear words in their animalistic snarling; in the confusion of melee, are they attacking with their swords or their foot-long canines? Heroes, particularly Dorasa and Estave, trumpet and bellow and crush their enemies beneath their feet. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The normal physical rules don't entirely apply on the hero plane. If a character explicitly invokes a myth to justify their actions, give them a bonus of +5 or so. If they continue to invoke the same myth successfully, increase the bonus or allow them to decrease their opponents' bonus. The same bonus applies to everyone else who invokes the same myth. Some mythlets are in the appendix; the conspirators are invoking the “Big Teeth kill the Intruders” myth. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Just as Earth defeats Storm in the elemental progression, so Earth cultists have an advantage over Storm cultists in the hero plane. Give Dorasa and Estave a bonus of +5 to resist all actions by an Orlanth cultist, rising to +10 when resisting magic with magic (alternatively, use the rules from Masters of Luck and Death, p. 52) |} At some point in this confrontation, if not before, the players of Gringle and Sarotar will want to reveal their true objectives and stab the other heroes in the back. Let them. The whole episode has been building up to this struggle between the player heroes, and its is their actions that will determine whether the peace process succeeds or not. ==== Scene II: Denouement ==== If the conspirators are victorious, they move on to ambush and kill Iristha and Mikhil. Normal life in the two clans is shattered as open, total war breaks out between them. The weaponthanes gain power and prestige, but at a dreadful cost. If the heroes are successful, they hear the sounds of Issaries and Lhankor Mhy passing below just as the conspirators are defeated. The heroes can return to the Singing Rocks in time for the wedding between Askos and Levru. If the heroes succeed, but Askos cannot or will not marry Levru, another bachelor can be found but the resulting peace treaty will be neither as strong nor as long-lasting. == Acknowledgements == I'd like to thank Rob Davies for his feedback from running and playtesting this scenario. I'd also like to thank Tim Ellis, Newt Newport, and Paul Kay for their insightful reviews and comments. == Appendix: Mythlets == The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll need a copy of this myth as a handout for play. === Orlanth Banishes The Thieves === One day, Orlanth came home from a day's work in the fields to find that his favourite neck torc was missing. He looked all over his stead for it: he lifted up the mattresses, he flew up into the rafters, he swirled into all the corners of the longhouse, but he couldn't find his neck torc anywhere. Later, at dinner, he heard Vadrus, his brother, boasting to his friends in the corner of the longhouse. Orlanth left his dinner to see what Vadrus was going on about this time. When he got there, he saw that Vadrus was wearing Orlanth's favourite neck torc! "That’s mine!" shouted Orlanth. "Give it back." "No," said Vadrus. "I ... found it, in the field. It’s mine now." "But my laws say that even when something is lost, it still belongs to its owner. It’s mine." "I claim it by right of conquest!" bellowed Vadrus. "If you want it, come an take it!" But instead of reaching for his sword, Orlanth reached for his lawstaff. "This is my law," he said, "and you will obey it!" And with that he cracked Vadrus's head with the staff and Vadrus dropped the torc. Vadrus and his friends ran out of the longhouse. Orlanth chased them all the way to the edge of his tula and told them never to come back. === Humakt Slays the Enemies === After Humakt had reclaimed Death, and after Orlanth had given him a place in his hall, Humakt faced many enemies. Whenever the Sky Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Night Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Plant Tribe attacked, Humakt killed their warriors and drove them back. But when others in the Storm Tribe attacked, no-one would fight them. Even thought the Storm Tribe were enemies, they were still kin and Orlanth would not attack them. Vadrus, and Valind, and even Ragnaglar came to Orlanth's tula to fight and steal, but Orlanth would not risk kinstrife by driving them off. Humakt said that he had no kin, and so could attack the enemies, but Orlanth always told him to stay in the longhouse. Humakt worried about this. He had given his word to Orlanth always to obey his commands, but he had also given his word to Orlanth always to defend his tula. Now his two oaths were in conflict. Humakt thought long and hard about this, and then decided what to do. Soon, the winds started to blow hot and cold and fierce. It was an omen that the bad members of the Storm Tribe were coming to attack again. As soon as he felt the wind, Humakt left Orlanth’s hall. That way, when Orlanth gave the orders not to attack Valind, Humakt wasn’t there to hear them. Humakt was free to carry out his oath to defend Orlanth’s tula and, when Valind stepped onto the tula, Humakt stepped up to him and killed him. === Ernalda Heals the Tribe === Even after Ernalda told the Orlanth to make the Storm Tribe, there were problems. Even after Ernalda told Orlanth what laws to make, there were problems. Even after Ernalda told Orlanth how to judge the lawsuits, there were problems. Ernalda sat and thought about the problems and what she could do about them. She discovered that the problems were always worse just after Issaries had visited, so the next time he came, she watched very carefully. She saw that Issaries had many beautiful and wonderful things he had found in foreign lands. She saw that everyone in the Tribe wanted all the things, but Issaries wouldn’t just give them away. No, Issaries traded for them, so that everyone had to give something up to get something new. This made a lot of people unhappy, because they weren't any better off after Issaries had been than they were before, and now they were angry that there were lots of things that Issaries had that they didn’t, and that some people had more things than others. Ernalda knew what to do. One night, when everyone was sleeping, she took away all the things that Issaries had brought and hid them in her secret place. The next morning when everyone woke up, they couldn't find the things Issaries had brought and they were all very angry. They all came to Orlanth’s hall and demanded that he did something about it. Then Ernalda spoke to them and showed them that they still had the most important thing of all, and that was the Storm Tribe. Everyone realised that Ernalda was speaking the truth, and they were all happy again. Then Ernalda took everything out of her secret place, and everyone agreed to share them among themselves. == Characters == [[Peace Process characters|Characters for this episode]] are on a separate page. If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. [[Category:HeroQuest|Peace Process]] [[Category:Glorantha|Peace Process]] [[Category:Scenarios|Peace Process]] 0fbd8ede8228153ccf59a6911514bbf8eb30a90d 1894 1885 2009-01-12T14:37:48Z Neil 2 Added links to pdfs wikitext text/x-wiki <blockquote>A [[:Category:HeroQuest|HeroQuest]] Episode by Neil Smith</blockquote> {|align="right" style="width:50%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Download formatted versions of the [[:Image:Peace-process-episode.pdf|scenario (PDF, 1.1Mb)]] and [[:Image:Peace-process-characters-ww.pdf|characters (PDF, 161Kb)]]. |} == Introduction == This is an episode for [[:Category:HeroQuest|HeroQuest]]. It was written to be played by five or six players (four at a pinch) in a single three to four hour session, typical of convention games. Six [[Peace Process characters|pre-generated characters]] are provided. It is designed as a drama, rather than a thriller: the engine of the scenario lies in the characters, their motivations, and the conflicts between them. The focus should be not so much on the events that unfold during the scenario as on how the heroes react and interact as a result of those events. I advise you to understand the inter-hero tensions fully before you run the scenario. Also, when running this episode, ensure that you give plenty of time for the players to chew the scenery and explore their heroes. This episode is set in an anonymous valley in the Quivin Mountains in Sartar. All the heroes are typical Orlanthi from two neighbouring clans as they attempt to bring an end to the feud that has damaged them for decades. This episode requires the '''HeroQuest''' core rulebook. '''Thunder Rebels''' and '''Storm Tribe''' are both extremely useful for background information. The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll probably need a copy of this myth as a handout for play. I'm told that '''Barbarian Adventures''' may come in handy, but I don't actually own it! {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Many contests are described in a short form, as '''''task''' (resistance): typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. |} == Backstory == {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Read this to the players at the start of the game. Don't mention that Sarotar and Gringle are conspiring against the other heroes. |} Until twenty years ago, the neighbouring Singing Rock and Hiord clans got on tolerably well; well enough for the young Singing Rock woman, '''Estave''', to marry '''Sarotar''', her true love, and leave her birth clan to join him in the Hiord clan. Soon after that, relations between the two clans began to break down and the usual cycle of cattle raids became more vicious on both sides. Eventually, a feud was declared and the fyrds skirmished. In that first battle, Brol the farmer was killed. '''Dorasa''', his wife, managed to have him resurrected but he had the Relife sickness. Brol turned to Humakt and became '''Mortality''', Champion of the Singing Rock clan. Since then, the feud has rumbled on, sometimes as a "cold war," sometimes erupting into violence. Recently, Estave and Dorasa have managed to broker a deal that will lead the two clans to peace. The two clans have agreed to end the feud and have charged Iristha (an Issarian from the Singing Rock clan) and Mikhil (a Lhankor Mhyite from the Hiord clan) to re-enact the "Issaries the Conciliator" myth. The quest will start in the Hirod tula tomorrow morning and finish atop the Singing Rocks tomorrow evening. The purpose of the quest is to discover the terms that will end the feud. When the heroquesters return, they will bring the resolution with them; both clans have promised to honour the terms on their return. The restoration of normal relations will be cemented by the marriage of '''Askos''' and Levru, a marriage arranged by Estave. Both clans have committed all their magical resources to this quest and, if either the quest or the marriage fail to happen tomorrow, the damage to both clans will be dreadful. The heroes have been charged with ensuring that the marriage takes place tomorrow evening, atop the Singing Rocks, when the heroquesters arrive. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |But all is not well. Sarotar and '''Gringle''', warleaders of the two opposing clans, have conspired to derail the quest by kidnapping Levru and ambushing the heroquesters on the Hero Plane. They each have their own reasons for this, but both agree that war is better than peace. |} == Summary == When the episode starts, all the heroes are at the edge of the Singing Rock tula, waiting for Levru to arrive. They are to form the honour guard for Levru as she travels to her new home there. Just as she arrives, her party is ambushed by a band of Gagarthi bandits (arranged by Sarotar), who kidnap Levru. The heroes will probably track the bandits, and may or may not discover their camp. They may be able to interrogate at least one Gagarthi and learn that they were hired to perform this ambush by someone in one of the clans. The heroes may try other ways of discovering more about the Gagarthi, but to little avail. The next day, during the Windsday rituals at the Singing Rocks, one of the clan's new weaponthanes is driven wild by impests and yavorlings and falls to his death from the Rocks. The heroes should soon discover that other weaponthanes are implicated in the plot to kidnap Levru. The heroes race to the weaponthanes' training ground where they discover Levru (unharmed) and a portal to the Hero Plane. The heroes step through the portal and foil an attempt by the conspirators to ambush and kill the heroquesters Iristha and Mikhil. At some point in all this, Gringle and Sarotar reveal themselves as the plot's ringleaders, probably as they attack the other heroes. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The players are almost certainly going to move a long way from the events described here. Let them. However, the characters carry the conflict with them (the important one is between Gringle & Sarotar and the rest of the heroes), and there is a tight deadline to work to, so they should bring themselves back on track. So long as you keep that in mind, you should be able to improvise interesting events around the heroes, whatever the players have them do. As examples, in one playtest, Sarotar confessed all just after the death at the Singing Rock, shopped Gringle (an NPC in that game) and asked for a chance to put right his mistakes: a very Orlanthi thing to do. In another session, the climax was when Mortality came to fulfil his vow to kill the defeated Sarotar, only for Estave to throw herself over her husband's body: Mortality killed them both. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Don't worry too much about the pacing in this episode. The heroes are likely to move through most of it quite quickly, but you're unlikely to finish too soon. The final scene, where Gringle , Sarotar, and the other conspirators stab the other heroes in the back, will take quite a while to play out. |} == The Episode == === Act I: Introduction === ==== Scene I: The Gagarthi Raid ==== ''Key Points: The Heroes get to strut their stuff, and some Gagarthi kidnap Levru.'' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |This scene is more about giving the players a chance to give their new characters a work out than influencing the plot. |} [[Image:Peace-process-map.png|right|thumb|400px|A map of the Singing Rock and Hiord tulas. This is an approximate map of the various locations of significance in this episode. The two tulas are in neighbouring valleys in the Quivin mountains. The land between them is mountainous and heavily forested, suitable only for hunting. The nearest the two tulas come to each other is at a pass between the high sheep pastures. ]] The scene opens with the heroes standing on the edge of the Singing Rock tula, waiting for Levru to arrive. They have all just left the Singing Rock stead where they have been preparing for Levru's arrival. Mortality has witnessed the Orlanthi Greeting that gave Sarotar the Blanket gift and Estave the Meat gift; this means that the Singing Rock clan has pledged hospitality and safety to both of them ('''Thunder Rebels''', pp. 34-36). The heroes are to be Levru's honour guard as she crosses the Singing Rock tula. She is being brought across the Hiord tula by an honour guard of Hiord weaponthanes. Just as the Hiord party comes into earshot, the heroes can use their (magical) perception abilities to '''''detect the Gagarthi waiting in ambush''' (3W): Spot ambush, sense gods nearby (-5)''. If they succeed, describe the feel of a hot, dry wind, the sand stinging their cheek, and the smell of hot flint. As soon as they think they have been rumbled, or when the Hiord party turn up, the Gagarthi attack. Many of them attack the Hiord party; the remainder try to hold back the heroes. The heroes face one Gagarthi leader per hero and two henchmen per hero , together with the odd whirlvish or two. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Gagarthi Leader: Sword & Shield Fighting 14W, (+3 auto-augment from Tough Outlaw = 5W), Walk on Winds 6W, Stinging Winds 2W, Tough Outlaw 5W, 39AP + 27AP + 27AP (from two henchmen) = 93AP. |- |Gagarthi Henchman: Sword & Shield Fighting 5W, Stinging Winds 18, Tough Outlaw 19; treat as followers, providing 27AP each. |- |Whirlvish: Abrade 10W, Spirit combat 15, 32AP. |} If the heroes win this encounter, they drive their assailants back into the Hiord party, who then capture and escape with Levru. If the heroes lose, they are forced back and fail to prevent the Gagarthi from kidnapping Levru. Try to ensure that none of the Gagarthi leaders are captured. ==== Scene II (Plot Turn I): Kidnap and Recriminations ==== ''Key Points: The heroes discover that Levru has been taken, and that the peace deal is not a foregone conclusion.'' When the Gagarthi leave, the heroes will want to investigate the Hiord party. They will discover that the 12 Hiordlings have been defeated, many of them are injured, and they have just noticed that Levru has been taken, snatched by a wind-walking Gagarthi. Many of the Hiordlings will blame the heroes for the attack, claiming that they were on Singing Rock lands and that the Singing Rocks are responsible for the injuries. Someone might even suggest that they wanted the Gagarthi to attack (though they have no evidence to support this: it's just hot-headed talking). Tempers can be calmed in a number of ways: tending to the wounded, boasting of how the Gagarthi were defeated, vows to avenge the terrible crime. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |As the heroes take stock of the situation, reinforce that the marriage between Levru and Askos is an essential part of the peace deal. If the marriage doesn't occur at the prescribed place and time, the feud will continue at great cost to both clans. |} === Act II: Conflict === ==== Scene I: Hunting the Wild Hunters ==== ''Key Points: The heroes discover that the Gagarthi had an informer from one of the two clans.'' The ball is now in the heroes' court. They are likely to want to gain some information about the Gagarthi and Levru: where they are, why they attacked, how they knew, and what are they planning for Levru. There are a few avenues that could be open to them. There could be Gagarthi prisoners to interrogate. Some Gagarthi fled on foot and can be tracked. The god-talker heroes may want to use divinations. Also, someone will have to tell the Singing Rock clan the bad news of Levru's kidnap. * ''Interrogating Gagarthi'': The bandits are fanatics, and all declared outlaw. They know that if they are ever brought to justice they are almost certain to be executed, so they have no real reason to co-operate. This makes interrogation difficult. The heroes know this (inform the players if need by), but they are likely anyway to want to interrogate them anyway. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate Gagarthi Bandits''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Resistance'': 12W2 |- |''Complete or Major Defeat'': The bandit either commits suicide or manages to snatch a hostage. |- |''Other Defeat or Tie'': The bandit reveals nothing. |- |''Marginal Victory'': The bandit hints that they knew that someone would be here. |- |''Minor Victory'': The bandits were waiting specifically for this group of Hiord clansfolk. |- |''Major Victory'': The Gagarthi were hired to kidnap “the girl.” |- |''Complete Victory'': This was definitely an inside job, connected to some big ritual. He doesn't know the identity of the insider. |} Heroes may '''''Recognise the Bandit''' (15W): Relationship to Clan'' and recall the bandit's name and deeds, and reduce the bandit's resistance according to the result of that contest. * ''Tracking the Gagarthi'': The wind-walking Gagarthi leaders are the ones who stole Levru, and they are difficult to track. However, the other Gagarthi who escaped are easier to follow: '''''Track the Gagarthi''' (3W): Track, Scout (-5), Singing Rock Geography (-10), Hiord Geography (-10)''. Any victory allows the heroes to follow the trail. It soon gets easier to follow once the wind-walking bandits come to earth. After a few hours of hard walking over some increasingly rough terrain (up and down some some very steep and heavily wooded slopes), the heroes come to the Gagarthi's (deserted) camp. The camp has been packed up recently and the firepit it still hot. Observant heroes will notice two fresh trails leaving the camp, one more heavily used than the other. Both peter out after a mile or so as they pass over bare, rocky ground in the fading light. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Their job done, the Gagarthi have been paid off and have left the area. Levru has been taken by some weaponthanes to the training camp in the Singing Rock tula. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |One potential problem point is persuading the heroes that they've lost the Gagarthi's trail. Stress the difficulty of following the trail, the worsening conditions, and time pressure the heroes are under to force them into trying another tack. |} * ''Asking the gods'': The heroes may ask the gods, or at least the Singing Rock clan wyter, where Levru and the Gagarthi are. (The Singing Rocks are the best bet, as most of the heroes have magical connections there they can call on.) Let the heroes go to town in describing the preparations, and involve the heroes in persuading the clan to support them and provide community support and objects to sacrifice. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Ask the Gods about Levru and the Gagarthi''' |- |''Appropriate abilities'': Pray to Orlanth, Pray to Ernalda, Devotee of <Deity> (-5) |- |''Modifiers and Augments'': Length of ritual (+3/hr), <Deity> Myths, various Relationships and Personality traits (for emotional involvement), various Relationships (for community support), auspicious location (e.g. the Singing Rocks) (+10), value of sacrifices (up to +10), not a member of the Singing Rock clan (-20), penalties for fatigue (contest based on time and exertion up to now). See also the Ritual Magic section, HeroQuest pp. 101-2. |- |''Resistance'': 10W3 |- |''Complete, Major, or Minor Defeat'': The heroes fail to learn anything. |- |''Marginal Defeat or Tie'': The Singing Rock wyter tells of a disturbance in the clan. |- |''Marginal Victory'': There is something wrong in the clan. Neither Levru nor the Gagarthi are touched by the wind or the earth within the Singing Rock tula. |- |''Minor Victory'': There is something wrong in the clan. Levru is within the tula; the Gagarthi are not. |- |''Major Victory'': There is discord in the Song of the Rocks, discord in the very heart of the tula. Levru is hidden within the tula. |- |''Complete Victory'': Kinstrife! Kinstrife! Kinstrife brought on by people both here and elsewhere! A vision of Levru being bound by a man with Singing Rock clan tatoos. |} By the time the rituals are complete, the heroes are liable to be too fatigued to do much more. They should go to bed disheartened that the peace deal is in danger of failure. ==== Scene II (Midpoint): When Impests Attack! ==== ''Key Points: The conspiracy is revealed when a weaponthane is driven to his death by Impests.'' [[Image:Peace-process-singing-rocks.png|right|thumb|400px|The Singing Rocks are the centre of the clan's lands. From the east, the ground rises up to a tall and increasingly narrow hill, rather like Castle Mount in Edinburgh. At the western end, the soil peters out to expose a jagged, rocky column over 150' high. As the wind blows around the hill, it swirls between the column and the cliff and sings and moans in honour of Orlanth and Ernalda. This swirling wind is the clan's wyter, and its song is understood by the chieftain. The peak of this hill is the holiest place in the tula, as it symbolises the marriage of Earth and Air; most rituals are led from here. A minor test of faith is to leap from the hill to the column; a major one is to leap from the column into the air, trusting that Orlanth will suspend you.]] At dawn the next morning, the heroes join the assembling clan atop the Singing Rocks. The mood is sombre as the clan contemplate the impending failure of the heroquest. The heroes may contemplate trying to persuade Sarostip that the quest should be abandoned, but they know that the consequences abandoning it would be worse than the consequences of failure, assuming that they could get word to Iristha nd Mikhil in time. The heroes may '''''notice that some weaponthanes are missing''' (2W): Relationship to warband, Relationship to clan (-8)'', though they won't necessarily know what that means. As the early morning worship starts, there is a scream from in the congregation as one of the weaponthanes starts to flail around wildly. Characters may '''''sense the clustering agents of reprisal''' (14): Sense gods nearby, Piety (-5)'' as the man is attacked by yavorlings and harulings (sparkers and impests) ('''Thunder Rebels''', pp. 81-82). As he attempts to flee, he runs blindly off the edge of the cliff and falls to smash on the rocks far below. Everyone is shocked, both because of the sudden and violent death, and for the calamitous omen it represents. The agents of reprisal follow him down and circle his corpse. They then rise again to swirl around the huddled clan. Perhaps their work is not yet done... ==== Scene III: The Conspiracy Uncovered ==== ''Key Points: The heroes discover that the conspiracy is led by the weaponthanes.'' All the Singing Rock heroes will recognise the dead man as Willem, a new weaponthane. They are likely to want to have pointed words with his colleagues, particularly in light of their suspicions that there is a traitor in the clan. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |'''Contest: Interrogate the Weaponthanes''' |- |''Appropriate abilities'': Persuade, Know Truth (-5), Control Children (-8), Fatherly (-10) |- |''Modifiers and Augments'': Threats of violence (-20 to +20, depending on contest v. resistance of 15W), promises of amnesty (as threats of violence), Relationship to warriors. |- |''Resistance'': 5W2 |- |''Complete or Major Defeat'': The militia manage to place the blame on the few Hiord clansmen present and incite a lynch mob. |- |''Other Defeat or Tie'': The weaponthanes reveal nothing. |- |''Marginal Victory'': Someone says that the weaponthanes have been plotting something, but there are no more details. |- |''Minor Victory'': A warrior with a mouthful of dirt blurts out that “They forced me to keep quiet!” and reveals that some weaponthanes (inlcuding Gringle, if he's an NPC) want the feud to continue. |- |''Major Victory'': The weaponthanes reveal that some of them have decided the feud should continue, and are conspiring to ensure that it does. |- |''Complete Victory'': The weaponthanes tell everything, including the involvement of Sarotar and Gringle, then attack them! |} ==== Scene IV (Plot Turn II): To the Training Ground ==== ''Key Points: The heroes rescue Levru and enter the Hero Plane.'' The heroes will either have evidence that the weaponthanes are involved, or suspect that this is the case. In either case, they are likely to want to pay a visit to their training ground, a mile or so from the stead. The training ground is a small clearing, about a hundred yards across, on a level piece of ground in the hills above the Singing Rocks. Around the field are archery butts, straw-stuffed dummies for sword practice, and other such paraphernalia. When the heroes arrive, the field is deserted. One one side of the field is the weaponthanes' longhouse. It is silent and dark, though smoke is rising from the smoke hole. The only doors are in the middle of the long side facing the field; they are closed and barred from the inside. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Play up the paranoia while the heroes are moving into the longhouse. They shouldn't know the place is deserted until they rush the longhouse. |} When the heroes enter the longhouse, three things are immediately apparent. First is the bullock in the middle of the floor, lying in a pool if its own blood. Second is Levru, gagged and tied to a chair at one end of the longhouse. Third is a large portal to the Hero Plane at the other end of the longhouse. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The portal looks rather like the Stargate from the TV series, if that would help your players visualise it. |} The bullock has been ritually slaughtered as part of a ceremony. Any god-talker will be able to deduce that the ritual was probably the one to open the portal. Levru is unharmed and will tell the heroes that, about an hour ago, a group of four weaponthanes opened the portal. They were talking about ambushing Iristha and Mikhil on the Hero Plane. Levru will tell the heroes that she recognised two of the weaponthanes as being Hiord clansmen; the remainder had Singing Rock clan tattoos. The heroes should realise that they are the only ones who can save the peace treaty, by eliminating the weaponthanes so that Iristha and Mikhil can complete their hero quest. The heroes will have to enter the hero plane. === Act III: Resolution === ==== Scene I: Rumble on the Hero Plane ==== ''Key Points: The heroes discover the full extent of the conspiracy, and it is either defeated or it isn't.'' When the heroes enter the hero plane, they can easily track the weaponthanes through a small patch of forest to where they are lying hidden above a trail, waiting to ambush Iristha and Mikhil when they pass through into the Big Teeths' lands. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Emphasise the hyper-realistic nature of the hero plane. Colours are more vibrant; vision is more acute; smells are more intense; sounds are more moving. |} The conspirators, Baranwulf, Illig, Kagrad, and Verlain (Ranged attack 16W, Close attack 6W2, Resist Magic 14W, Hardened Consiprator 15W, all after augments) will attack the heroes if attacked themselves, or if they are being beaten in a battle of wits and words. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The hero plane is the world of myth, and you should bring this out in your desciptions. When the conspirators attack, they are taking the part of the Long Teeth; their leather armour regrows stiff, spotted fur; it's difficult to hear words in their animalistic snarling; in the confusion of melee, are they attacking with their swords or their foot-long canines? Heroes, particularly Dorasa and Estave, trumpet and bellow and crush their enemies beneath their feet. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |The normal physical rules don't entirely apply on the hero plane. If a character explicitly invokes a myth to justify their actions, give them a bonus of +5 or so. If they continue to invoke the same myth successfully, increase the bonus or allow them to decrease their opponents' bonus. The same bonus applies to everyone else who invokes the same myth. Some mythlets are in the appendix; the conspirators are invoking the “Big Teeth kill the Intruders” myth. |} {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Just as Earth defeats Storm in the elemental progression, so Earth cultists have an advantage over Storm cultists in the hero plane. Give Dorasa and Estave a bonus of +5 to resist all actions by an Orlanth cultist, rising to +10 when resisting magic with magic (alternatively, use the rules from Masters of Luck and Death, p. 52) |} At some point in this confrontation, if not before, the players of Gringle and Sarotar will want to reveal their true objectives and stab the other heroes in the back. Let them. The whole episode has been building up to this struggle between the player heroes, and its is their actions that will determine whether the peace process succeeds or not. ==== Scene II: Denouement ==== If the conspirators are victorious, they move on to ambush and kill Iristha and Mikhil. Normal life in the two clans is shattered as open, total war breaks out between them. The weaponthanes gain power and prestige, but at a dreadful cost. If the heroes are successful, they hear the sounds of Issaries and Lhankor Mhy passing below just as the conspirators are defeated. The heroes can return to the Singing Rocks in time for the wedding between Askos and Levru. If the heroes succeed, but Askos cannot or will not marry Levru, another bachelor can be found but the resulting peace treaty will be neither as strong nor as long-lasting. == Acknowledgements == I'd like to thank Rob Davies for his feedback from running and playtesting this scenario. I'd also like to thank Tim Ellis, Newt Newport, and Paul Kay for their insightful reviews and comments. == Appendix: Mythlets == The major mythic resource for the scenario is ''Issaries the Conciliator'' ('''Storm Tribe''', pp. 121-2). You'll need a copy of this myth as a handout for play. === Orlanth Banishes The Thieves === One day, Orlanth came home from a day's work in the fields to find that his favourite neck torc was missing. He looked all over his stead for it: he lifted up the mattresses, he flew up into the rafters, he swirled into all the corners of the longhouse, but he couldn't find his neck torc anywhere. Later, at dinner, he heard Vadrus, his brother, boasting to his friends in the corner of the longhouse. Orlanth left his dinner to see what Vadrus was going on about this time. When he got there, he saw that Vadrus was wearing Orlanth's favourite neck torc! "That’s mine!" shouted Orlanth. "Give it back." "No," said Vadrus. "I ... found it, in the field. It’s mine now." "But my laws say that even when something is lost, it still belongs to its owner. It’s mine." "I claim it by right of conquest!" bellowed Vadrus. "If you want it, come an take it!" But instead of reaching for his sword, Orlanth reached for his lawstaff. "This is my law," he said, "and you will obey it!" And with that he cracked Vadrus's head with the staff and Vadrus dropped the torc. Vadrus and his friends ran out of the longhouse. Orlanth chased them all the way to the edge of his tula and told them never to come back. === Humakt Slays the Enemies === After Humakt had reclaimed Death, and after Orlanth had given him a place in his hall, Humakt faced many enemies. Whenever the Sky Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Night Tribe attacked, Humakt killed their warriors and drove them back. Whenever the Plant Tribe attacked, Humakt killed their warriors and drove them back. But when others in the Storm Tribe attacked, no-one would fight them. Even thought the Storm Tribe were enemies, they were still kin and Orlanth would not attack them. Vadrus, and Valind, and even Ragnaglar came to Orlanth's tula to fight and steal, but Orlanth would not risk kinstrife by driving them off. Humakt said that he had no kin, and so could attack the enemies, but Orlanth always told him to stay in the longhouse. Humakt worried about this. He had given his word to Orlanth always to obey his commands, but he had also given his word to Orlanth always to defend his tula. Now his two oaths were in conflict. Humakt thought long and hard about this, and then decided what to do. Soon, the winds started to blow hot and cold and fierce. It was an omen that the bad members of the Storm Tribe were coming to attack again. As soon as he felt the wind, Humakt left Orlanth’s hall. That way, when Orlanth gave the orders not to attack Valind, Humakt wasn’t there to hear them. Humakt was free to carry out his oath to defend Orlanth’s tula and, when Valind stepped onto the tula, Humakt stepped up to him and killed him. === Ernalda Heals the Tribe === Even after Ernalda told the Orlanth to make the Storm Tribe, there were problems. Even after Ernalda told Orlanth what laws to make, there were problems. Even after Ernalda told Orlanth how to judge the lawsuits, there were problems. Ernalda sat and thought about the problems and what she could do about them. She discovered that the problems were always worse just after Issaries had visited, so the next time he came, she watched very carefully. She saw that Issaries had many beautiful and wonderful things he had found in foreign lands. She saw that everyone in the Tribe wanted all the things, but Issaries wouldn’t just give them away. No, Issaries traded for them, so that everyone had to give something up to get something new. This made a lot of people unhappy, because they weren't any better off after Issaries had been than they were before, and now they were angry that there were lots of things that Issaries had that they didn’t, and that some people had more things than others. Ernalda knew what to do. One night, when everyone was sleeping, she took away all the things that Issaries had brought and hid them in her secret place. The next morning when everyone woke up, they couldn't find the things Issaries had brought and they were all very angry. They all came to Orlanth’s hall and demanded that he did something about it. Then Ernalda spoke to them and showed them that they still had the most important thing of all, and that was the Storm Tribe. Everyone realised that Ernalda was speaking the truth, and they were all happy again. Then Ernalda took everything out of her secret place, and everyone agreed to share them among themselves. == Characters == [[Peace Process characters|Characters for this episode]] are on a separate page. If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. [[Category:HeroQuest|Peace Process]] [[Category:Glorantha|Peace Process]] [[Category:Scenarios|Peace Process]] 6c413b50edc29d15113cdf8e86db19e943167a24 14 1555 1872 2009-01-08T21:11:31Z Neil 2 New page: Pages related to roleplaying game scenarios [[Category:RPGs]] wikitext text/x-wiki Pages related to roleplaying game scenarios [[Category:RPGs]] 0e9dbdfcea8a86e4ee83885446f0e38aa74a9553 0 1480 1873 1622 2009-01-08T21:12:25Z Neil 2 Added Scenario category tag wikitext text/x-wiki Gareth Hanrahan's scenario ''[http://www.gameeire.com/Scenarios/bplanet.html Treacherous Waters]'' is a wonderful blood opera, but the files don't include the statistics for the pregenerated characters. Here are my versions of them. Note that: # These are slightly underpowered Exceptional characters: they have the skill packages of Exceptional characters, but the attributes of Everyday characters; # Superscripts on skill ratings show the number of dice to roll for that skill (I can't keep track of aptitudes when I'm running the game); # Only exceptional equipment is listed here: common-or-garden equipment, including guns and armour, is available on the station. == Cmdr Alan Baum == * ''Packages'': Origin: GEO; Backgrounds: GEO, University; Professional: Expert Espionage (Analyst), Specialist Military. * ''Primary Attributes'': Build -1, Fitness 0, Agility 0, Dexterity 0, Awareness +1 (Chemical 0, Hearing +1, Intuition +2, Touch 0, Vision +2), Intellect +2, Presence -1, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 5; Aquatics 5, Throwing 4; Armed Combat 2, Unarmed Combat 4; Leadership 2&sup2;, Logistics 2&sup2;, Strategy 4&sup2;, Tactics 2&sup2;; Persuasion 5&sup3;, Writing 4&sup3;; Culture (Earth) 5&sup2;, Culture (GEO) 9&sup2;, Culture (Incorporate) 6&sup2;, Culture (Military) 3&sup2;; Handguns 4, Longarms 3; History 2, Politics 6; First Aid 2; Heavy Weapons 2; Hiding 2&sup2;, Shadowing 2&sup2;, Sneaking 2&sup2;; Bribery 2&sup2;, Disguise 2&sup2;, Fast Talk 4&sup2;, Forgery 1&sup2;, Lockpicking 2&sup2;; Computers 6&sup3;, Demolitions 1&sup3;, Electronics 6&sup3;, Remote Operations 4&sup3;. * ''Mods and equipment'': Diver genmod, neural jack, heavy vest, combat helmet. == Dr Faith Tourmanlene == * ''Packages'': Origin: Earth Orbit; Backgrounds: GEO, University; Professional: Expert Science, Novice Admin, Novice Survival. * ''Primary Attributes'': Build -2, Fitness -1, Agility -2, Dexterity 0, Awareness 0 (Chemical 0, Hearing -1, Intuition -1, Touch +1, Vision +1), Intellect +3, Presence +1, Will +1. * ''Secondary Attributes'': Endurance 0, Reflexes -1, Strength -1, Toughness -1. * ''Skills'': Bureaucracy 6&sup2;, Economics 1&sup2;, Law 1&sup2;, Management 1&sup2;; Aquatics 1, Freefall 1, Throwing 2; Leadership 1, Logistics 1; Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 4&sup3;, Writing 3&sup3;; Culture (Earth) 5&sup2;, Culture (GEO) 4&sup2;, Culture (Incorporate) 3&sup2;, Culture (Native) 2&sup2;; Longarms 2; History 2, Politics 3; Biochemistry 7&sup3;, Botany 7&sup3;, Genetics 5&sup3;, Zoology 6&sup3;; First Aid 1; Astronomy 3&sup2;, Chemistry 3&sup2;, Geology 3&sup2;, Meterology 4&sup2;, Physics 3&sup2;; Hiding 1, Sneaking 2; Fast Talk 2; Fishing 2, Foraging 2, Mountaineering 2, Navigation 1, Orienteering 2; Computers 7&sup2;, Damage Control 1&sup2;, Electronics 1&sup2;, Mechanics 1&sup2;, Remote Operations 3&sup2;; Piloting 2. * ''Mods and equipment'': Neural jack, anti-poison, salt tolerance, multiglands. == Dr Tsung-Li == * ''Packages'': Origin: Freezone&emdash;Wasteland; Backgrounds: Incorporate, University; Professional: Expert Science, Novice Espoinage, Novice Technical. * ''Primary Attributes'': Build -2, Fitness 0, Agility -1, Dexterity 0, Awareness 0 (Chemical -1, Hearing 0, Intuition +1, Touch +1, Vision -1), Intellect +2, Presence -1, Will +1. * ''Secondary Attributes'': Endurance 0, Reflexes 0, Strength -1, Toughness 0. * ''Skills'': Bureaucracy 5, Economics 1, Law 1; Unarmed Combat 1; Persuasion 4&sup2;, Writing 4&sup2;; Culture (Earth) 5&sup2;, Culture (Incorporate) 3&sup2;, Culture (Military) 2&sup2;; Handguns 1, Longarms 1; History 1, Politics 2; Biochemistry 3&sup2;, Botany 3&sup2;, Genetics 3&sup2;, Zoology 3&sup2;; First Aid 1; Astronomy 5&sup3;, Chemistry 5&sup3;, Geology 5&sup3;, Meterology 7&sup3;, Physics 7&sup3;; Hiding 1, Shadowing 1, Sneaking 1; Bribery 1, Disguise 1, Fast Talk 4, Lockpicking 3; Foraging 1, Navigation 3, Orienteering 1, Tracking 1; Computers 8&sup3;, Damage Contol 2&sup3;, Electronics 3&sup3;, Mechanics 3&sup3;, Remote Operations 6&sup3;; Driving 2, Piloting 2. * ''Mods and equipment'': Uplink neural jack, implanted computer, audio/visual bug suite. == Lt Feyd Al-Umarj == * ''Packages'': Origin: Mars; Backgrounds: GEO, Colonial; Professional: Expert Military (Marine), Specialist Survival. * ''Primary Attributes'': Build +1, Fitness +2, Agility +2, Dexterity +1, Awareness 0 (Chemical 0, Hearing +1, Intuition -1, Touch 0, Vision 0 [+2 in low light]), Intellect -2, Presence 0, Will -1. * ''Secondary Attributes'': Endurance +1, Reflexes +2, Strength +1, Toughness +1. * ''Skills'': Bureaucracy 2; Aquatics 7&sup2;, Parachuting 2&sup2;, Throwing 4&sup2;; Armed Combat 6&sup3;, Unarmed Combat 5&sup3;; Leadership 3, Strategy 3, Tactics 3; Culture (Cetacean) 1, Culture (Colonial) 3, Culture (GEO) 4, Culture (Incorporate) 1, Culture (Military) 5, Culture (Native) 1, Culture (Spacer) 3; Handguns 6&sup3;, Longarms 7&sup3;; History 1, Politics 1; First Aid 5; Heavy Weapons 5; Meterology 2; Hiding 3&sup2;, Sneaking 5&sup2;; Fishing 3, Foraging 3, Mountaineering 3, Navigation 3, Orienteering 4, Tracking 3; Computers 2, Damage Control 1, Demolitions 1, Electronics 2, Mechanics 1, Remote Operations 3; Driving 3&sup2;, Piloting 2&sup2;, Sailing 3&sup2;. * ''Mods and equipment'': Diver genmod, accelerated neurons, infrared sensors, subdermal plates, adrenal shunt, salt tolerance, heavy vest, combat helmet. == Dr Vlad Olyah == * ''Packages'': Origin: GEO; Backgrounds: GEO, University; Professional: Specialist Science, Novice Admin, Novice Survival, Novice Arts & Entertainment. * ''Primary Attributes'': Build -1, Fitness +1, Agility 0, Dexterity 0, Awareness 0 (Chemical -1, Hearing +1, Intuition 0, Touch +1, Vision -1), Intellect +1, Presence +1, Will 0. * ''Secondary Attributes'': Endurance +1, Reflexes 0, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 52, Economics 12, Law 12; Aquatics 4; Leadership 1, Logistics 1; Interspec 3&sup3;, Negotiation 4&sup3;, Oration 1&sup3;, Persuasion 3&sup3;, Writing 5&sup3;; Culture (Earth) 7&sup2;, Culture (GEO) 7&sup2;, Culture (Incorporate) 2&sup2;, Culture (Native) 4&sup2;; Acting 5, Dance 3, Music 6; Longarms 1; History 2, Politics 4; Biochemistry 4&sup3;, Botany 5&sup3;, Genetics 4&sup3;, Zoology 5&sup3;; First Aid 1; Astronomy 2&sup2;, Chemistry 2&sup2;, Geology 2&sup2;, Meterology 3&sup2;, Physics 2&sup2;; Hiding 1, Sneaking 2; Fast Talk 2; Fishing 2, Foraging 2, Mountaineering 2, Navigation 1, Orienteering 2, Tracking 2; Computers 7&sup2;, Mechanics 1&sup2;, Remote Operations 2&sup2;. * ''Mods and equipment'': Diver genmod, perfect pitch, multiglands. == Martha Grosse == * ''Packages'': Origin: Urban; Backgrounds: Incorporate, Independent; Professional: Expert Espionage (Operative), Novice Entertainment, Novice Survival. * ''Primary Attributes'': Build -1, Fitness +2, Agility +3, Dexterity +2, Awareness 0 (Chemical -1, Hearing +1, Intuition 0, Touch -1, Vision +1), Intellect -1, Presence -2, Will 0. * ''Secondary Attributes'': Endurance +2, Reflexes +2, Strength 0, Toughness 0. * ''Skills'': Bureaucracy 3, Economics 3, Management 1; Aquatics 5; Unarmed Combat 3&sup2;; Negotiation 6&sup2;, Persuasion 6&sup2;; Culture (Earth) 6, Culture (GEO) 2, Culture (Incorporate) 6, Culture (Street) 2; Acting 5, Dance 3, Photography 3; Handguns 3, Longarms 3; Politics 2; Botany 1, Zoology 1; First Aid 2; Meterology 1; Hiding 5&sup3;, Shadowing 3&sup3;, Sneaking 3&sup3;; Bribery 3&sup3;, Disguise 2&sup3;, Fast Talk 5&sup3;, Lockpicking 3&sup3;; Fishing 2, Foraging 2, Mountaineering 2, Navigation 1, Orienteering 2, Tracking 2; Computers 7&sup2;, Demolitions 3&sup2;, Electronics 3&sup2;, Remote Operations 3&sup2;; Driving 4&sup2;, Piloting 3&sup2;. * ''Mods and equipment'': Diver genmod, implanted sensory recorder, accelerated neurons, neural jack. == Native Ecoterrorists == * Reflexes 0, Agility 0, Dexterity 0, Toughness +1, Fitness +1, Will +1, Armour 4. * SMG 3&sup2;, DR 6/9. * Assault Rifle 5&sup2;, DR 11. * Grenade 5&sup2;, DR 8. * Knife 3&sup3;, DR 2. [[Category:Blue Planet]] [[Category:Scenarios]] dcba023832b20a6454595437a2014250d43bdff0 6 1556 1875 2009-01-08T21:24:21Z Neil 2 Character map for [[The Peace Process]] scenario wikitext text/x-wiki Character map for [[The Peace Process]] scenario aa317715e0769e3e93f35a65bc2f29c87a4c89b2 1879 1875 2009-01-08T21:32:52Z Neil 2 wikitext text/x-wiki Map of [[Peace Process characters|characters and relationships]] for [[The Peace Process]] scenario. [[Category:Scenarios]] 5d26433d9c114dde52bb77f79ddf38a861e44637 6 1557 1876 2009-01-08T21:29:07Z Neil 2 Map for [[The Peace Process]] scenario wikitext text/x-wiki Map for [[The Peace Process]] scenario 89d80acbcbbc35fd8c8f4d52ba16e6998b837300 1878 1876 2009-01-08T21:31:41Z Neil 2 wikitext text/x-wiki Map for [[The Peace Process]] scenario. [[Category:Scenarios]] 381feef04c64f7dd24b7ed42db1489fd5800f18c 1882 1878 2009-01-08T21:46:01Z Neil 2 wikitext text/x-wiki Map for [[The Peace Process]] scenario. This is an approximate map of the various locations of significance in this episode. The two tulas are in neighbouring valleys in the Quivin mountains. The land between them is mountainous and heavily forested, suitable only for hunting. The nearest the two tulas come to each other is at a pass between the high sheep pastures. [[Category:Scenarios]] 766069e5c283f58b2d0fa6d510ed22b346b6fe33 1889 1882 2009-01-08T22:18:37Z Neil 2 uploaded a new version of "[[Image:Peace-process-map.png]]" wikitext text/x-wiki Map for [[The Peace Process]] scenario. This is an approximate map of the various locations of significance in this episode. The two tulas are in neighbouring valleys in the Quivin mountains. The land between them is mountainous and heavily forested, suitable only for hunting. The nearest the two tulas come to each other is at a pass between the high sheep pastures. [[Category:Scenarios]] 766069e5c283f58b2d0fa6d510ed22b346b6fe33 6 1558 1877 2009-01-08T21:30:15Z Neil 2 A diagram of the Singing Rocks for [[The Peace Process]] scenario [[Category:Scenarios]] wikitext text/x-wiki A diagram of the Singing Rocks for [[The Peace Process]] scenario [[Category:Scenarios]] 3178f5b3823b47adbb89a3687f025c6bb9431323 1880 1877 2009-01-08T21:34:35Z Neil 2 wikitext text/x-wiki A picture of the Singing Rocks for [[The Peace Process]] scenario [[Category:Scenarios]] aacb738d9a05ac44b4c0755065b3d0e49ce60b02 1886 1880 2009-01-08T21:54:38Z Neil 2 wikitext text/x-wiki A picture of the Singing Rocks for [[The Peace Process]] scenario. The Singing Rocks are the centre of the clan's lands. From the east, the ground rises up to a tall and increasingly narrow hill, rather like Castle Mount in Edinburgh. At the western end, the soil peters out to expose a jagged, rocky column over 150' high. As the wind blows around the hill, it swirls between the column and the cliff and sings and moans in honour of Orlanth and Ernalda. This swirling wind is the clan's wyter, and its song is understood by the chieftain. The peak of this hill is the holiest place in the tula, as it symbolises the marriage of Earth and Air; most rituals are led from here. A minor test of faith is to leap from the hill to the column; a major one is to leap from the column into the air, trusting that Orlanth will suspend you. [[Category:Scenarios]] ceee9c85c3595af455eaafc1950d61ac89044d32 0 1559 1887 2009-01-08T22:09:37Z Neil 2 New page: These are the pre-generated characters for [[The Peace Process]] episode for [[Category:HeroQuest|HeroQuest]]. [[Image:Peace-process-character-map.png|Peace Process characters relationshi... wikitext text/x-wiki These are the pre-generated characters for [[The Peace Process]] episode for [[Category:HeroQuest|HeroQuest]]. [[Image:Peace-process-character-map.png|Peace Process characters relationships]] This is a simple relationship map showing the player heroes and significant NPCs in this episode If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. == Askos (Singing Rock Clan) == Oh, to be young, healthy, and in love! Life is good, and you intend to go out there, grab it by the horns, and squeeze every last experience from it. You worship, no, you live your life as, Destor the Adventurer. You are dynamic, proud, vigourous, active. You move as often and as quickly as the winds. No-one can make you do anything! You have mixed feelings about the peace deal. You’re disappointed that it’s on the rug, because the clashes with the Hiord clan are a great and dashing adventure. On the other hand, things are starting to get a bit boring as you’ve come up against all the Hiord warriors several times already. The peace deal will have some benefits, though. Your aunt Estave (now married into the Hiord clan) has arranged for the deal to be cemented by your marriage to Levru, and she’s gorgeous. Your current lover, Dorasa (ex-wife of Mortality the clan champion; he doesn’t seem to mind, though you were almost hoping he would) is getting a bit wrinkled now, though she’s great in bed. Also, peace with the Hiords will mean that you’re free to take up new challenges! Huzzah! Heortling 19 Farming Heortling Customs Heortling Myths Sartar Geography Warrior 7W Endurance Fyrd Combat Guard Camp Hide in Cover Javelin Fighting Know Hiord Clan Tactics Listen 3WW Recognize Fighting Style Recognize Foe Scout Spear and Shield Fighting Spot Ambush Sprint 18W Stay Awake 18 Sword and Shield Fighting Devotee of Destor (Orlanth Adventurous) 14W Boast Brew Woad Dodge Attack Jump 2WW Orlanth Myths Sense Change in Weather Thunder Brother Myths Combat 16W (Aid Throw With Wind, Armour of Woad, Enchant Silver ritual, Flickering Blade, Leaping Shield, Overbear Foe, Sword Help) Movement 2WW (Burst of Speed, Leap Over Obstacle, Leap Through Window, Run On Mud, Run up Cliffs, Sunset Leap; Find a Way, Lift Object With Winds) Wind 11W (Call Clouds, Call Wind, Drive Away Clouds, Hear Words on Wind, Summon Umbroli) Initiate of Niskis (Orlanth Adventurous) 18 Sensuality Personality Active 14W Boastful 4W Brave 2W Easily Distracted 19 Intensely Emotional 2W Noble 19 Proud 4W Relationships Betrothed to Levru 16 Lover of Dorasa 3W Nephew of Estave 6W Rival of Mortality 18 Singing Rock Clan 19 Singing Rock Warband 19 Susurrate the Umbroli 19 (Strong 8W, Throw Debris 2W), living in a silver neck torc Other Abilities Climb 5W Neatly Trimmed Beard 16 Quick Wits 18 Shout “Huzzah!” 18 Swing 17W Twinkling Eye 18 Whisper Sweet Nothings 2W Winning Smile 19 3 Hero Points Mortality (Singing Rock Clan) You used to be Brol the Farmer, until you were caught up in the first raid by the Hiord Clan, and killed. Your wife, Dorasa, managed to beg a Chalana Arroy priestess to resurrect you, but she couldn’t bring you all back. Some essential part of you, your vitality, stayed on the Other Side. Life had no meaning, so you drifted to the worship of Humakt, Death, who gave you a course to steer by. You had your second funeral when you gave your new life to Humakt and left your old life behind. You became a warrior and eventually the clan’s Swordthane (champion), and fought in many battles, mostly against the Hiord clan.. You bear them no rancour. When your ex-wife took Askos as a lover, your were disturbed more than you should have been. You have no real opinion on the peace settlement. Your chieftain has given his backing to the deal, and you have sworn to obey the chieftain in all things. Despite that, you feel some unease that warriors on both sides have committed sins against the other, and no punishment will be imposed for them. You dislike Gringle, the fyrd leader. He is too passive, too conservative. That is not the way to win battles. But he spurns all your advice and your offers to assist in the fyrd’s training. Heortling 16 Farming Heortling Customs Heortling Myths Warrior 16W Endurance Fyrd Combat Guard Camp Hide in Cover Javelin Fighting Know Hiord Clan Tactics Listen Recognize Fighting Style Recognize Foe Sartar Geography Scout Spear and Shield Fighting Spot Ambush Sprint Stay Awake Sword and Shield Fighting 8WW Weaponthane 3W Ambush 3W [Forbidden by Geas] Axe and Shield Fighting Climb Hide in Cover 19 Riding Champion 9W Brawling 2W Greatsword Fighting 4WW Ignore Pain Know Neighbouring Warriors See Opponent’s Weakness Single Sword Fighting 18W Strong Thrown Knife Fighting 8W Tough Personality Brave 17 Determined 18 Dispassionate 14W Fearless 8W Grim 8W Professional 18 Honourable 4W Devotee of Makla Mann (Humakt) 4W Craft Weapon Dagger Fighting 12W Fyrd Tactics Humakt Myths Recognize Lie Sense Assassin Death 19 (Bless Corpse, Death Song Berserk, Fight Undead, Lay Ghost, Visage of Fear) Honour 8W (Empower Oath, Know Truth, Sense Ambush, Sever Relationship, Shame Coward) Loyalty 6W (Fight to the Death, Inspire Loyalty, Sense Disloyalty, Stand Loyal) [Makla Mann’s Blessing: Never participate in ambush.] [Cannot receive healing from Chalana Arroy healers] Relationships Ex-Husband of Dorasar 6W Jealous of Askos 3W Legally Dead 10WW Loyal to Singing Rock Clan 8W Rival of Gringle 2W Singing Rock Clan 18 Singing Rock Warband 2W 3 Hero Points Gringle (Singing Rock Clan) You are the clan’s Shieldthane, one of the Four Fighting Storms. You lead the fyrd, the militia formed from all the clan’s fighting men. You’ve held this position for decades, and you know exactly what’s going on. As the fyrd leader, you’re responsible for turning the young hot-headed boys into reliable and steadfast fighting men. And you’re good at it. One thing that makes you good at it is that you’ve learnt that what doesn’t kill us makes us stronger. That’s why you don’t want the feud with the Hiord clan to end. If the feud ends, there will be no opportunity left for the youngsters to test themselves and become strong. It’s strong young men that we’ll all need for the struggle to kick out the Lunars, not the soft layabouts they’ll become without you. You’ve come to know and respect many of the Hiord weaponthanes, particularly Sarotar. He’s fed you details of what will be happening when the young bride Levru appears, and you’ve arranged for the Gagarthi to kidnap her. After that, your job is to keep the searchers in the wrong place until your accomplices can fatally sabotage the Issaries heroquest. This may not be obeying your chieftain, your chosen leader, but it’s for the good of everyone. The fact that continuing the feud will continue your position of influence is all rather incidental. You also don’t like Mortality, the clan champion. He keeps trying to meddle with the fyrd and their training, trying to get them to charge and attack more. You know that’s just going to get them killed. Heortling 2W Farming Heortling Customs Heortling Myths Warrior 16W Endurance Fyrd Combat 4WW Guard Camp Hide in Cover Javelin Fighting Know Hiord Clan Tactics Listen Recognize Fighting Style Recognize Foe Sartar Geography Scout Singing Rock Geography 3WW Spear and Shield Fighting Sprint 2W Stay Awake Sword and Shield Fighting Weaponthane 5W Ambush 16W Axe and Shield Fighting Climb 2W Hide in Cover Riding Warband Leader 2WW Businesslike Fighter Fyrd Tactics 5WW Shieldwall Fighting Spot Ambush Initiate of Skarkval (Orlanth Allfather) 3W Orlanth Myths Sense Change in Weather Allfather Combat 8W Making Common Magic 19 Dry Clothes, Heal Flesh, Hide in the Open, Keep Wind Out, Light Fire Personality Brave 17 Dedicated 19 Dislike cocky young men 3W Fatherly 6W Hard-Working 2W Professional 18W Responsible 5W Relationships Co-conspirator with Sarotar 18W Hate Hiord Clan 4W Respect of Hiord Warband 4W Rival of Mortality 18 Singing Rock Clan 18W Singing Rock Warband 3WW Four Storms: Olend (Swordthane) 19 (Warrior 3W, Finovan 19) Sartark (Shieldthane) 12W (Warrior 4W, Starkval 19) Govoran (Spearthane) 6W (Archer 2W, Helamakt 18) Janerra (Backboy) 8W (Healer 12W, Bevara 8W) Others Belittle Warrior 15W Boast 6W Displace Blame 4W Gammy Leg 18 Misdirect Question 7W Seen it All Before 17W Sharp Words 2W Spread Rumours 2W 3 Hero Points Dorasa (Singing Rock Clan) You represent Ernalda in the Singing Rock Clan's inner ring. As such, it's your job to make sure that the men-folk see what the sensible thing is and then do it, rather than swanning off on some other trivial but exciting distraction. In any situation, it's obvious to you what the right thing to do is. That's the easy part. What's hard is making everyone else see it, and then getting them to do it. Life would be so much easier if people just let you make the decisions, and they did what you told them. That way, you could have the whole clan pulling in the same direction and you could achieve so much more. The problem you've got at the moment is this peace deal with the Hiord clan. It's the right thing to do: that's obvious. That's why you've negotiated the deal in the first place. The cost in lives, cattle, limbs, and heartache is enormous and a real distraction from the real business of growing and thriving. The trouble is, many of the menfolk seem to enjoy the fighting and the raiding instead of staying at home ploughing and herding. You don't think that everyone is as supportive of your initiative as they should be. The only downside to the deal is that it means you should stop having sex with Askos, the youngster you took as a lover when your husband died and came back as Mortality the Humakti. That's a shame, because is energetic and inventive in bed. But, the peace deal gets cemented with his marriage to Levru from the Hiord clan. Technically, adultery is a crime, but you're sure you can persuade him to keep you as a lover. Heortling 2W Heortling Customs Heortling Myths Housework Sartar Geography Weaving Steadwife 8W Butchering Domestic Chores First Aid Gardening Husbandry Maintain Peace in Stead Manage Household Preserve Food Threshing Weaving God-Talker 16W Initiate to Ernalda Perform Sacrifice Pray to Ernalda Sense Gods Nearby Sense Spirits Nearby Priest 5W Chant for Hours Dance Sacred Dances Ernaldan Philosophy Lead Worship Devotee of Kadone (Ernalda the Queen) 9W Ernalda Myths Meaningful Glare 7WW Orate Persuade 17W Earth Queen 4W (Banish Air Daimon, Command Earth Animal, Command Earth Daimon, Dismiss Air Elemental, Dismiss Earth Elemental, Exorcise Air Spirit Exorcise Earth Spirit, Enchant Copper ritual, Summon Talosi) Find Another Way 11W (Calm Anger, Create Ring, Rally Women, Shame Husband, Stop Argument, Think of the Children) Heaviness 12W (Force Flyer to Ground, Keep Object on Ground, Make Person Heavy, Make Object Heavy, Realization of Responsibility, Show Error of Ways, Stop Fleeing) Relationships Blue Spice Clan (birth clan) 4W Clod the Talosi 2W (Large 2W, Strong 8W, Swallow Victim 4W) Ex-wife of Mortality 18 Lover of Askos 2W Singing Rock Clan 12W Singing Rock Inner Ring Member 14W Singing Rock Loomhouse 2WW Work with Estave 18 Personality Doesn't Suffer Fools Gladly 11W Just 3W Loving 18 Motherly 16 Pious 14W Pragmatic 19W Strong-Willed 4WW Unmerciful to Enemies 19 Others Be Heard 19 Cutting Insult 18W Ignore Man 12W Stand Ground 15W Understand Consequences 11W Work Together 6W 3 Hero Points Estave (Hiord Clan) The feud between your two clans is just tragic. You'd just left the Singing Rock clan to marry Sarotar in the Hiord Clan when the feud started. The feud meant that you weren't able to go back to visit your family in the Singing Rock Clan and you were left to make your own way in the suddenly hostile Hiord clan. You decided to tough it out, but it has been your earnest wish for years to find some resolution to the feud. This ambition hasn't always been popular in the Hiord clan, especially with Sarotar, but you've kept at it. And now it seems that the end is in sight. You've found Levru, a good, sweet girl, and persuaded her to marry your nephew Askos. The marriage will be the final step in the reconcilliation between your two clans. Sarotar is a good and loving husband, but he just doesn't see the feud in the same way that you do. There's been a lot of bad things done by both clans in the feud, and Sarotar now thinks that the Singing Rock clan cannot be trusted. His attitude has caused a lot of ill feeling recently and you're now not sure if your marriage will survive the reconciliation. You're having second thoughts about whether it's really worth it. Heortling 18 Heortling Customs Heortling Myths Housework Sartar Geography Weaving Steadwife 4W Butchering Domestic Chores First Aid Gardening Husbandry Maintain Peace in Stead Manage Household Preserve Food Threshing Weaving Matchmaker 18W Judge Compatibility Judge Personality 4WW Know Genealogy Know Local Bloodlines Negotiate Marriage Pass Surreptitious Message Sense Real Motive God-Talker 4W Initiate to Ernalda Perform Sacrifice Pray to Ernalda Sense Gods Nearby Sense Spirits Nearby Devotee of Vela (Ernalda Allmother) 13W Animal Lore Control Children Domestic Chores Ernalda Myths Plant Lore Bless Family 14W (Beautify Self, Comfort Sick Child, Conceal Family, Ease Fears, Make Husband Laugh) Bless Mothers 9W (Aid Conception, Aid Contraception, Bless Birth, Bless Courtship, Bless Domestic Animal, Bless Pregnancy) Matchmaking 16W (Ask Ancestors for Blessing, Assess Boy, Assess Girl, Calm Ardour, Ignite Passion, Reconcile Parents) Relationships Aunt of Askos 3W Grateful Families 4W Hiord Clan 7W Hiord Clan Loomhouse 13W Maternal to Levru 19 Singing Rock Clan 18 Wife of Sarotar 6WW Work with Dorasar 2W Personality Generous 16W Loving 4WW Motherly 17W Pious 12W Unmerciful to Enemies 18 Others Calm Emotions 14W Gossip 17W Look Harmless 4W Nag 4WW Pass Unseen 18 See the Best in People 9W 3 Hero Points Sarotar (Hiord Clan) You’ve been defending the Hiord Clan against those bastards in the Singing Rock clan for years now. During that time, you’ve risen to a position of prominence in the clan. Your devotion to Rigsdal, the Night Watchman, and your dependability, have ensured that the clan has come to rely on you to keep them safe. You like being in that position, as it means that you’re listened to by the clan, and your opinions carry weight. All that is set to change. If the peace deal goes ahead, there will be peace between the clans. You won’t be needed to stand guard every night, and you’re opinions won’t be treated so seriously in the clan moots. You’ll be a no-one. The worst thing about it is that your wife, Estave, has been instrumental in setting up the deal, and she went ahead with it despite you commanding her not to. Without her work in making the first contacts and then setting up the marriage between Levru and Askos, the reconciliation would never be about to happen. And if she’s not around, the deal may not happen still. That’s why you contacted Gringle to arrange a way to stop things. That’s why you contacted the Gagarthi to kidnap Levru on her way to the Singing Rock tula. That’s why you’re considering doing something truly awful to Estave, if that’s what’s needed. Heortling 2W Farming Heortling Customs 18W Heortling Myths Warrior 18W Endurance Guard Camp Know Singing Rock Clan Tactics Sartar Geography Hide in Cover Stay Awake Spot Ambush 12W Listen Spear and Shield Fighting Sword and Shield Fighting Javelin Fighting Recognize Fighting Style Recognize Foe Sprint Scout Fyrd Combat Weaponthane 11W Ambush Axe and Shield Fighting Climb Hide in Cover Riding God-Talker 7W Initiate to Orlanth Perform Sacrifice Pray to Orlanth Sense Gods Nearby Sense Spirits Nearby Devotee of Rigsdal (Orlanth Allfather) 6W Orlanth Myths Sense Change in Weather Endure Cold 12W Keen Hearing 8W Keen Vision Alert 7WW Allfather 6W (Lead by Example, Protect Home, Silencing Bellow, Summon Family) Making 2W (Acclaim Chieftain, Bless New Building, Bless Tools, Oversee Marriage ritual, Work Hard) Vigilance 8W (Hear Any Sound, Ignore Distraction, Night Vision, See Far, Sleep With One Eye Open) Relationships Husband to Estave 2W Hiord Clan 9W Hiord Clan Warband 13W Co-conspirator with Gringle 7W Gagarthi Bandits 18 Personality Loving Husband 18 Brave 16W Boastful 15 Determined 5W Fatherly 18 Hard-Working 7W Responsible 18W Reliable 7WW Resolute 11W Pious 17 Desire Power 18W Others Lie 14W Be Heard in Moot 4W Persuade Others 3W Give Commands 2WW 3 Hero Points [[Category:Scenarios]] [[Category:HeroQuest]] 9967063f480cfd23449ec43738536f21116346e7 1890 1887 2009-01-12T14:19:28Z Neil 2 wikitext text/x-wiki These are the pre-generated characters for [[The Peace Process]] episode for [[Category:HeroQuest|HeroQuest]]. __TOC__ == Character relationship map == [[Image:Peace-process-character-map.png|Peace Process characters relationships]] This is a simple relationship map showing the player heroes and significant NPCs in this episode If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. == Askos (Singing Rock Clan) == Oh, to be young, healthy, and in love! Life is good, and you intend to go out there, grab it by the horns, and squeeze every last experience from it. You worship, no, you live your life as, Destor the Adventurer. You are dynamic, proud, vigourous, active. You move as often and as quickly as the winds. No-one can make you do anything! You have mixed feelings about the peace deal. You’re disappointed that it’s on the rug, because the clashes with the Hiord clan are a great and dashing adventure. On the other hand, things are starting to get a bit boring as you’ve come up against all the Hiord warriors several times already. The peace deal will have some benefits, though. Your aunt Estave (now married into the Hiord clan) has arranged for the deal to be cemented by your marriage to Levru, and she’s gorgeous. Your current lover, Dorasa (ex-wife of Mortality the clan champion; he doesn’t seem to mind, though you were almost hoping he would) is getting a bit wrinkled now, though she’s great in bed. Also, peace with the Hiords will mean that you’re free to take up new challenges! Huzzah! {| |- valign="top" |width="33%"|'''Heortling 19''' * Farming * Heortling Customs * Heortling Myths * Sartar Geography '''Warrior 7W''' * Endurance * Fyrd Combat * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen 3WW * Recognize Fighting Style * Recognize Foe * Scout * Spear and Shield Fighting * Spot Ambush * Sprint 18W * Stay Awake 18 * Sword and Shield Fighting |width="34%"|'''Devotee of Destor (Orlanth Adventurous) 14W''' * Boast * Brew Woad * Dodge Attack * Jump 2WW * Orlanth Myths * Sense Change in Weather * Thunder Brother Myths * ''Combat'' 16W (Aid Throw With Wind, Armour of Woad, Enchant Silver ritual, Flickering Blade, Leaping Shield, Overbear Foe, Sword Help) * ''Movement'' 2WW (Burst of Speed, Leap Over Obstacle, Leap Through Window, Run On Mud, Run up Cliffs, Sunset Leap; Find a Way, Lift Object With Winds) * ''Wind'' 11W (Call Clouds, Call Wind, Drive Away Clouds, Hear Words on Wind, Summon Umbroli) '''Initiate of Niskis (Orlanth Adventurous) 18''' * Sensuality |width="33%"|'''Personality''' * Active 14W * Boastful 4W * Brave 2W * Easily Distracted 19 * Intensely Emotional 2W * Noble 19 * Proud 4W '''Relationships''' * Betrothed to Levru 16 * Lover of Dorasa 3W * Nephew of Estave 6W * Rival of Mortality 18 * Singing Rock Clan 19 * Singing Rock Warband 19 * Susurrate the Umbroli 19 (Strong 8W, Throw Debris 2W), living in a silver neck torc '''Other Abilities''' * Climb 5W * Neatly Trimmed Beard 16 * Quick Wits 18 * Shout “Huzzah!” 18 * Swing 17W * Twinkling Eye 18 * Whisper Sweet Nothings 2W * Winning Smile 19 '''3 Hero Points''' |} == Mortality (Singing Rock Clan) == You used to be Brol the Farmer, until you were caught up in the first raid by the Hiord Clan, and killed. Your wife, Dorasa, managed to beg a Chalana Arroy priestess to resurrect you, but she couldn’t bring you all back. Some essential part of you, your vitality, stayed on the Other Side. Life had no meaning, so you drifted to the worship of Humakt, Death, who gave you a course to steer by. You had your second funeral when you gave your new life to Humakt and left your old life behind. You became a warrior and eventually the clan’s Swordthane (champion), and fought in many battles, mostly against the Hiord clan.. You bear them no rancour. When your ex-wife took Askos as a lover, your were disturbed more than you should have been. You have no real opinion on the peace settlement. Your chieftain has given his backing to the deal, and you have sworn to obey the chieftain in all things. Despite that, you feel some unease that warriors on both sides have committed sins against the other, and no punishment will be imposed for them. You dislike Gringle, the fyrd leader. He is too passive, too conservative. That is not the way to win battles. But he spurns all your advice and your offers to assist in the fyrd’s training. {| |- valign="top" |width="33%"|'''Heortling 16''' * Farming * Heortling Customs * Heortling Myths '''Warrior 16W''' * Endurance * Fyrd Combat * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen * Recognize Fighting Style * Recognize Foe * Sartar Geography * Scout * Spear and Shield Fighting * Spot Ambush * Sprint * Stay Awake * Sword and Shield Fighting 8WW '''Weaponthane 3W''' * Ambush 3W [Forbidden by Geas] * Axe and Shield Fighting * Climb * Hide in Cover 19 * Riding |width="33%"|'''Champion 9W''' * Brawling 2W * Greatsword Fighting 4WW * Ignore Pain * Know Neighbouring Warriors * See Opponent’s Weakness * Single Sword Fighting 18W * Strong * Thrown Knife Fighting 8W * Tough '''Personality''' * Brave 17 * Determined 18 * Dispassionate 14W * Fearless 8W * Grim 8W * Professional 18 * Honourable 4W |width="34%"|'''Devotee of Makla Mann (Humakt) 4W''' * Craft Weapon * Dagger Fighting 12W * Fyrd Tactics * Humakt Myths * Recognize Lie * Sense Assassin * ''Death'' 19 (Bless Corpse, Death Song Berserk, Fight Undead, Lay Ghost, Visage of Fear) * ''Honour'' 8W (Empower Oath, Know Truth, Sense Ambush, Sever Relationship, Shame Coward) * ''Loyalty'' 6W (Fight to the Death, Inspire Loyalty, Sense Disloyalty, Stand Loyal) * [Makla Mann’s Blessing: Never participate in ambush.] * [Cannot receive healing from Chalana Arroy healers] '''Relationships''' * Ex-Husband of Dorasar 6W * Jealous of Askos 3W * Legally Dead 10WW * Loyal to Singing Rock Clan 8W * Rival of Gringle 2W * Singing Rock Clan 18 * Singing Rock Warband 2W '''3 Hero Points''' |} == Gringle (Singing Rock Clan) == You are the clan’s Shieldthane, one of the Four Fighting Storms. You lead the fyrd, the militia formed from all the clan’s fighting men. You’ve held this position for decades, and you know exactly what’s going on. As the fyrd leader, you’re responsible for turning the young hot-headed boys into reliable and steadfast fighting men. And you’re good at it. One thing that makes you good at it is that you’ve learnt that what doesn’t kill us makes us stronger. That’s why you don’t want the feud with the Hiord clan to end. If the feud ends, there will be no opportunity left for the youngsters to test themselves and become strong. It’s strong young men that we’ll all need for the struggle to kick out the Lunars, not the soft layabouts they’ll become without you. You’ve come to know and respect many of the Hiord weaponthanes, particularly Sarotar. He’s fed you details of what will be happening when the young bride Levru appears, and you’ve arranged for the Gagarthi to kidnap her. After that, your job is to keep the searchers in the wrong place until your accomplices can fatally sabotage the Issaries heroquest. This may not be obeying your chieftain, your chosen leader, but it’s for the good of everyone. The fact that continuing the feud will continue your position of influence is all rather incidental. You also don’t like Mortality, the clan champion. He keeps trying to meddle with the fyrd and their training, trying to get them to charge and attack more. You know that’s just going to get them killed. {| |- valign="top" |width="33%"|'''Heortling 2W''' * Farming * Heortling Customs * Heortling Myths '''Warrior 16W''' * Endurance * Fyrd Combat 4WW * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen * Recognize Fighting Style * Recognize Foe * Sartar Geography * Scout * Singing Rock Geography 3WW * Spear and Shield Fighting * Sprint 2W * Stay Awake * Sword and Shield Fighting '''Weaponthane 5W''' * Ambush 16W * Axe and Shield Fighting * Climb 2W * Hide in Cover * Riding |width="33%"|'''Warband Leader 2WW''' * Businesslike Fighter * Fyrd Tactics 5WW * Shieldwall Fighting * Spot Ambush '''Initiate of Skarkval (Orlanth Allfather) 3W''' * Orlanth Myths * Sense Change in Weather * ''Allfather'' * ''Combat 8W'' * ''Making'' '''Common Magic 19''' * Dry Clothes, Heal Flesh, Hide in the Open, Keep Wind Out, Light Fire '''Personality''' * Brave 17 * Dedicated 19 * Dislike cocky young men 3W * Fatherly 6W * Hard-Working 2W * Professional 18W * Responsible 5W |width="34%"|'''Relationships''' * Co-conspirator with Sarotar 18W * Hate Hiord Clan 4W * Respect of Hiord Warband 4W * Rival of Mortality 18 * Singing Rock Clan 18W * Singing Rock Warband 3WW * Four Storms: :Olend (Swordthane) 19 (Warrior 3W, Finovan 19) :Sartark (Shieldthane) 12W (Warrior 4W, Starkval 19) :Govoran (Spearthane) 6W (Archer 2W, Helamakt 18) :Janerra (Backboy) 8W (Healer 12W, Bevara 8W) '''Others''' * Belittle Warrior 15W * Boast 6W * Displace Blame 4W * Gammy Leg 18 * Misdirect Question 7W * Seen it All Before 17W * Sharp Words 2W * Spread Rumours 2W '''3 Hero Points''' |} == Dorasa (Singing Rock Clan) == You represent Ernalda in the Singing Rock Clan's inner ring. As such, it's your job to make sure that the men-folk see what the sensible thing is and then do it, rather than swanning off on some other trivial but exciting distraction. In any situation, it's obvious to you what the right thing to do is. That's the easy part. What's hard is making everyone else see it, and then getting them to do it. Life would be so much easier if people just let you make the decisions, and they did what you told them. That way, you could have the whole clan pulling in the same direction and you could achieve so much more. The problem you've got at the moment is this peace deal with the Hiord clan. It's the right thing to do: that's obvious. That's why you've negotiated the deal in the first place. The cost in lives, cattle, limbs, and heartache is enormous and a real distraction from the real business of growing and thriving. The trouble is, many of the menfolk seem to enjoy the fighting and the raiding instead of staying at home ploughing and herding. You don't think that everyone is as supportive of your initiative as they should be. The only downside to the deal is that it means you should stop having sex with Askos, the youngster you took as a lover when your husband died and came back as Mortality the Humakti. That's a shame, because is energetic and inventive in bed. But, the peace deal gets cemented with his marriage to Levru from the Hiord clan. Technically, adultery is a crime, but you're sure you can persuade him to keep you as a lover. {| |- valign="top" |width="33%"|'''Heortling Heortling 2W''' * Heortling Customs * Heortling Myths * Housework * Sartar Geography * Weaving '''Steadwife 8W''' * Butchering * Domestic Chores * First Aid * Gardening * Husbandry * Maintain Peace in Stead * Manage Household * Preserve Food * Threshing * Weaving '''God-Talker 16W''' * Initiate to Ernalda * Perform Sacrifice * Pray to Ernalda * Sense Gods Nearby * Sense Spirits Nearby * Priest 5W * Chant for Hours * Dance Sacred Dances * Ernaldan Philosophy * Lead Worship |width="34%"|'''Devotee of Kadone (Ernalda the Queen) 9W''' * Ernalda Myths * Meaningful Glare 7WW * Orate * Persuade 17W * ''Earth Queen'' 4W (Banish Air Daimon, Command Earth Animal, Command Earth Daimon, Dismiss Air Elemental, Dismiss Earth Elemental, Exorcise Air Spirit Exorcise Earth Spirit, Enchant Copper ritual, Summon Talosi) * ''Find Another Way'' 11W (Calm Anger, Create Ring, Rally Women, Shame Husband, Stop Argument, Think of the Children) * ''Heaviness'' 12W (Force Flyer to Ground, Keep Object on Ground, Make Person Heavy, Make Object Heavy, Realization of Responsibility, Show Error of Ways, Stop Fleeing) |width="33%"|'''Relationships''' * Blue Spice Clan (birth clan) 4W * Clod the Talosi 2W (Large 2W, Strong 8W, Swallow Victim 4W) * Ex-wife of Mortality 18 * Lover of Askos 2W * Singing Rock Clan 12W * Singing Rock Inner Ring Member 14W * Singing Rock Loomhouse 2WW * Work with Estave 18 '''Personality''' * Doesn't Suffer Fools Gladly 11W * Just 3W * Loving 18 * Motherly 16 * Pious 14W * Pragmatic 19W * Strong-Willed 4WW * Unmerciful to Enemies 19 '''Others''' * Be Heard 19 * Cutting Insult 18W * Ignore Man 12W * Stand Ground 15W * Understand Consequences 11W * Work Together 6W '''3 Hero Points''' |} == Estave (Hiord Clan) == The feud between your two clans is just tragic. You'd just left the Singing Rock clan to marry Sarotar in the Hiord Clan when the feud started. The feud meant that you weren't able to go back to visit your family in the Singing Rock Clan and you were left to make your own way in the suddenly hostile Hiord clan. You decided to tough it out, but it has been your earnest wish for years to find some resolution to the feud. This ambition hasn't always been popular in the Hiord clan, especially with Sarotar, but you've kept at it. And now it seems that the end is in sight. You've found Levru, a good, sweet girl, and persuaded her to marry your nephew Askos. The marriage will be the final step in the reconcilliation between your two clans. Sarotar is a good and loving husband, but he just doesn't see the feud in the same way that you do. There's been a lot of bad things done by both clans in the feud, and Sarotar now thinks that the Singing Rock clan cannot be trusted. His attitude has caused a lot of ill feeling recently and you're now not sure if your marriage will survive the reconciliation. You're having second thoughts about whether it's really worth it. {| |- valign="top" |width="33%"|'''Heortling 18''' * Heortling Customs * Heortling Myths * Housework * Sartar Geography * Weaving '''Steadwife 4W''' * Butchering * Domestic Chores * First Aid * Gardening * Husbandry * Maintain Peace in Stead * Manage Household * Preserve Food * Threshing * Weaving '''Matchmaker 18W''' * Judge Compatibility * Judge Personality 4WW * Know Genealogy * Know Local Bloodlines * Negotiate Marriage * Pass Surreptitious Message * Sense Real Motive |width="34%"|'''God-Talker 4W''' * Initiate to Ernalda * Perform Sacrifice * Pray to Ernalda * Sense Gods Nearby * Sense Spirits Nearby '''Devotee of Vela (Ernalda Allmother) 13W''' * Animal Lore * Control Children * Domestic Chores * Ernalda Myths * Plant Lore * ''Bless Family'' 14W (Beautify Self, Comfort Sick Child, Conceal Family, Ease Fears, Make Husband Laugh) * ''Bless Mothers'' 9W (Aid Conception, Aid Contraception, Bless Birth, Bless Courtship, Bless Domestic Animal, Bless Pregnancy) * ''Matchmaking'' 16W (Ask Ancestors for Blessing, Assess Boy, Assess Girl, Calm Ardour, Ignite Passion, Reconcile Parents) |width="33%"|'''Relationships''' * Aunt of Askos 3W * Grateful Families 4W * Hiord Clan 7W * Hiord Clan Loomhouse 13W * Maternal to Levru 19 * Singing Rock Clan 18 * Wife of Sarotar 6WW * Work with Dorasar 2W '''Personality''' * Generous 16W * Loving 4WW * Motherly 17W * Pious 12W * Unmerciful to Enemies 18 '''Others''' * Calm Emotions 14W * Gossip 17W * Look Harmless 4W * Nag 4WW * Pass Unseen 18 * See the Best in People 9W '''3 Hero Points''' |} == Sarotar (Hiord Clan) == You’ve been defending the Hiord Clan against those bastards in the Singing Rock clan for years now. During that time, you’ve risen to a position of prominence in the clan. Your devotion to Rigsdal, the Night Watchman, and your dependability, have ensured that the clan has come to rely on you to keep them safe. You like being in that position, as it means that you’re listened to by the clan, and your opinions carry weight. All that is set to change. If the peace deal goes ahead, there will be peace between the clans. You won’t be needed to stand guard every night, and you’re opinions won’t be treated so seriously in the clan moots. You’ll be a no-one. The worst thing about it is that your wife, Estave, has been instrumental in setting up the deal, and she went ahead with it despite you commanding her not to. Without her work in making the first contacts and then setting up the marriage between Levru and Askos, the reconciliation would never be about to happen. And if she’s not around, the deal may not happen still. That’s why you contacted Gringle to arrange a way to stop things. That’s why you contacted the Gagarthi to kidnap Levru on her way to the Singing Rock tula. That’s why you’re considering doing something truly awful to Estave, if that’s what’s needed. {| |- valign="top" |width="33%"|'''Heortling 2W''' * Farming * Heortling Customs 18W * Heortling Myths '''Warrior 18W''' * Endurance * Guard Camp * Know Singing Rock Clan Tactics * Sartar Geography * Hide in Cover * Stay Awake * Spot Ambush 12W * Listen * Spear and Shield Fighting * Sword and Shield Fighting * Javelin Fighting * Recognize Fighting Style * Recognize Foe * Sprint * Scout * Fyrd Combat '''Weaponthane 11W''' * Ambush * Axe and Shield Fighting * Climb * Hide in Cover * Riding |width="34%"|'''God-Talker 7W''' * Initiate to Orlanth * Perform Sacrifice * Pray to Orlanth * Sense Gods Nearby * Sense Spirits Nearby '''Devotee of Rigsdal (Orlanth Allfather) 6W''' * Orlanth Myths * Sense Change in Weather * Endure Cold 12W * Keen Hearing 8W * Keen Vision * Alert 7WW * ''Allfather'' 6W (Lead by Example, Protect Home, Silencing Bellow, Summon Family) * ''Making'' 2W (Acclaim Chieftain, Bless New Building, Bless Tools, Oversee Marriage ritual, Work Hard) * ''Vigilance'' 8W (Hear Any Sound, Ignore Distraction, Night Vision, See Far, Sleep With One Eye Open) '''Relationships''' * Husband to Estave 2W * Hiord Clan 9W * Hiord Clan Warband 13W * Co-conspirator with Gringle 7W * Gagarthi Bandits 18 |width="33%"|'''Personality''' * Loving Husband 18 * Brave 16W * Boastful 15 * Determined 5W * Fatherly 18 * Hard-Working 7W * Responsible 18W * Reliable 7WW * Resolute 11W * Pious 17 * Desire Power 18W '''Others''' * Lie 14W * Be Heard in Moot 4W * Persuade Others 3W * Give Commands 2WW '''3 Hero Points''' |} [[Category:Scenarios]] [[Category:HeroQuest]] [[Category:Glorantha]] 2996621f26caa3ce06239c9d134f390c72b32d00 1891 1890 2009-01-12T14:21:06Z Neil 2 Redorderd page wikitext text/x-wiki These are the pre-generated characters for [[The Peace Process]] episode for [[:Category:HeroQuest|HeroQuest]]. __TOC__ If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. == Askos (Singing Rock Clan) == Oh, to be young, healthy, and in love! Life is good, and you intend to go out there, grab it by the horns, and squeeze every last experience from it. You worship, no, you live your life as, Destor the Adventurer. You are dynamic, proud, vigourous, active. You move as often and as quickly as the winds. No-one can make you do anything! You have mixed feelings about the peace deal. You’re disappointed that it’s on the rug, because the clashes with the Hiord clan are a great and dashing adventure. On the other hand, things are starting to get a bit boring as you’ve come up against all the Hiord warriors several times already. The peace deal will have some benefits, though. Your aunt Estave (now married into the Hiord clan) has arranged for the deal to be cemented by your marriage to Levru, and she’s gorgeous. Your current lover, Dorasa (ex-wife of Mortality the clan champion; he doesn’t seem to mind, though you were almost hoping he would) is getting a bit wrinkled now, though she’s great in bed. Also, peace with the Hiords will mean that you’re free to take up new challenges! Huzzah! {| |- valign="top" |width="33%"|'''Heortling 19''' * Farming * Heortling Customs * Heortling Myths * Sartar Geography '''Warrior 7W''' * Endurance * Fyrd Combat * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen 3WW * Recognize Fighting Style * Recognize Foe * Scout * Spear and Shield Fighting * Spot Ambush * Sprint 18W * Stay Awake 18 * Sword and Shield Fighting |width="34%"|'''Devotee of Destor (Orlanth Adventurous) 14W''' * Boast * Brew Woad * Dodge Attack * Jump 2WW * Orlanth Myths * Sense Change in Weather * Thunder Brother Myths * ''Combat'' 16W (Aid Throw With Wind, Armour of Woad, Enchant Silver ritual, Flickering Blade, Leaping Shield, Overbear Foe, Sword Help) * ''Movement'' 2WW (Burst of Speed, Leap Over Obstacle, Leap Through Window, Run On Mud, Run up Cliffs, Sunset Leap; Find a Way, Lift Object With Winds) * ''Wind'' 11W (Call Clouds, Call Wind, Drive Away Clouds, Hear Words on Wind, Summon Umbroli) '''Initiate of Niskis (Orlanth Adventurous) 18''' * Sensuality |width="33%"|'''Personality''' * Active 14W * Boastful 4W * Brave 2W * Easily Distracted 19 * Intensely Emotional 2W * Noble 19 * Proud 4W '''Relationships''' * Betrothed to Levru 16 * Lover of Dorasa 3W * Nephew of Estave 6W * Rival of Mortality 18 * Singing Rock Clan 19 * Singing Rock Warband 19 * Susurrate the Umbroli 19 (Strong 8W, Throw Debris 2W), living in a silver neck torc '''Other Abilities''' * Climb 5W * Neatly Trimmed Beard 16 * Quick Wits 18 * Shout “Huzzah!” 18 * Swing 17W * Twinkling Eye 18 * Whisper Sweet Nothings 2W * Winning Smile 19 '''3 Hero Points''' |} == Mortality (Singing Rock Clan) == You used to be Brol the Farmer, until you were caught up in the first raid by the Hiord Clan, and killed. Your wife, Dorasa, managed to beg a Chalana Arroy priestess to resurrect you, but she couldn’t bring you all back. Some essential part of you, your vitality, stayed on the Other Side. Life had no meaning, so you drifted to the worship of Humakt, Death, who gave you a course to steer by. You had your second funeral when you gave your new life to Humakt and left your old life behind. You became a warrior and eventually the clan’s Swordthane (champion), and fought in many battles, mostly against the Hiord clan.. You bear them no rancour. When your ex-wife took Askos as a lover, your were disturbed more than you should have been. You have no real opinion on the peace settlement. Your chieftain has given his backing to the deal, and you have sworn to obey the chieftain in all things. Despite that, you feel some unease that warriors on both sides have committed sins against the other, and no punishment will be imposed for them. You dislike Gringle, the fyrd leader. He is too passive, too conservative. That is not the way to win battles. But he spurns all your advice and your offers to assist in the fyrd’s training. {| |- valign="top" |width="33%"|'''Heortling 16''' * Farming * Heortling Customs * Heortling Myths '''Warrior 16W''' * Endurance * Fyrd Combat * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen * Recognize Fighting Style * Recognize Foe * Sartar Geography * Scout * Spear and Shield Fighting * Spot Ambush * Sprint * Stay Awake * Sword and Shield Fighting 8WW '''Weaponthane 3W''' * Ambush 3W [Forbidden by Geas] * Axe and Shield Fighting * Climb * Hide in Cover 19 * Riding |width="33%"|'''Champion 9W''' * Brawling 2W * Greatsword Fighting 4WW * Ignore Pain * Know Neighbouring Warriors * See Opponent’s Weakness * Single Sword Fighting 18W * Strong * Thrown Knife Fighting 8W * Tough '''Personality''' * Brave 17 * Determined 18 * Dispassionate 14W * Fearless 8W * Grim 8W * Professional 18 * Honourable 4W |width="34%"|'''Devotee of Makla Mann (Humakt) 4W''' * Craft Weapon * Dagger Fighting 12W * Fyrd Tactics * Humakt Myths * Recognize Lie * Sense Assassin * ''Death'' 19 (Bless Corpse, Death Song Berserk, Fight Undead, Lay Ghost, Visage of Fear) * ''Honour'' 8W (Empower Oath, Know Truth, Sense Ambush, Sever Relationship, Shame Coward) * ''Loyalty'' 6W (Fight to the Death, Inspire Loyalty, Sense Disloyalty, Stand Loyal) * [Makla Mann’s Blessing: Never participate in ambush.] * [Cannot receive healing from Chalana Arroy healers] '''Relationships''' * Ex-Husband of Dorasar 6W * Jealous of Askos 3W * Legally Dead 10WW * Loyal to Singing Rock Clan 8W * Rival of Gringle 2W * Singing Rock Clan 18 * Singing Rock Warband 2W '''3 Hero Points''' |} == Gringle (Singing Rock Clan) == You are the clan’s Shieldthane, one of the Four Fighting Storms. You lead the fyrd, the militia formed from all the clan’s fighting men. You’ve held this position for decades, and you know exactly what’s going on. As the fyrd leader, you’re responsible for turning the young hot-headed boys into reliable and steadfast fighting men. And you’re good at it. One thing that makes you good at it is that you’ve learnt that what doesn’t kill us makes us stronger. That’s why you don’t want the feud with the Hiord clan to end. If the feud ends, there will be no opportunity left for the youngsters to test themselves and become strong. It’s strong young men that we’ll all need for the struggle to kick out the Lunars, not the soft layabouts they’ll become without you. You’ve come to know and respect many of the Hiord weaponthanes, particularly Sarotar. He’s fed you details of what will be happening when the young bride Levru appears, and you’ve arranged for the Gagarthi to kidnap her. After that, your job is to keep the searchers in the wrong place until your accomplices can fatally sabotage the Issaries heroquest. This may not be obeying your chieftain, your chosen leader, but it’s for the good of everyone. The fact that continuing the feud will continue your position of influence is all rather incidental. You also don’t like Mortality, the clan champion. He keeps trying to meddle with the fyrd and their training, trying to get them to charge and attack more. You know that’s just going to get them killed. {| |- valign="top" |width="33%"|'''Heortling 2W''' * Farming * Heortling Customs * Heortling Myths '''Warrior 16W''' * Endurance * Fyrd Combat 4WW * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen * Recognize Fighting Style * Recognize Foe * Sartar Geography * Scout * Singing Rock Geography 3WW * Spear and Shield Fighting * Sprint 2W * Stay Awake * Sword and Shield Fighting '''Weaponthane 5W''' * Ambush 16W * Axe and Shield Fighting * Climb 2W * Hide in Cover * Riding |width="33%"|'''Warband Leader 2WW''' * Businesslike Fighter * Fyrd Tactics 5WW * Shieldwall Fighting * Spot Ambush '''Initiate of Skarkval (Orlanth Allfather) 3W''' * Orlanth Myths * Sense Change in Weather * ''Allfather'' * ''Combat 8W'' * ''Making'' '''Common Magic 19''' * Dry Clothes, Heal Flesh, Hide in the Open, Keep Wind Out, Light Fire '''Personality''' * Brave 17 * Dedicated 19 * Dislike cocky young men 3W * Fatherly 6W * Hard-Working 2W * Professional 18W * Responsible 5W |width="34%"|'''Relationships''' * Co-conspirator with Sarotar 18W * Hate Hiord Clan 4W * Respect of Hiord Warband 4W * Rival of Mortality 18 * Singing Rock Clan 18W * Singing Rock Warband 3WW * Four Storms: :Olend (Swordthane) 19 (Warrior 3W, Finovan 19) :Sartark (Shieldthane) 12W (Warrior 4W, Starkval 19) :Govoran (Spearthane) 6W (Archer 2W, Helamakt 18) :Janerra (Backboy) 8W (Healer 12W, Bevara 8W) '''Others''' * Belittle Warrior 15W * Boast 6W * Displace Blame 4W * Gammy Leg 18 * Misdirect Question 7W * Seen it All Before 17W * Sharp Words 2W * Spread Rumours 2W '''3 Hero Points''' |} == Dorasa (Singing Rock Clan) == You represent Ernalda in the Singing Rock Clan's inner ring. As such, it's your job to make sure that the men-folk see what the sensible thing is and then do it, rather than swanning off on some other trivial but exciting distraction. In any situation, it's obvious to you what the right thing to do is. That's the easy part. What's hard is making everyone else see it, and then getting them to do it. Life would be so much easier if people just let you make the decisions, and they did what you told them. That way, you could have the whole clan pulling in the same direction and you could achieve so much more. The problem you've got at the moment is this peace deal with the Hiord clan. It's the right thing to do: that's obvious. That's why you've negotiated the deal in the first place. The cost in lives, cattle, limbs, and heartache is enormous and a real distraction from the real business of growing and thriving. The trouble is, many of the menfolk seem to enjoy the fighting and the raiding instead of staying at home ploughing and herding. You don't think that everyone is as supportive of your initiative as they should be. The only downside to the deal is that it means you should stop having sex with Askos, the youngster you took as a lover when your husband died and came back as Mortality the Humakti. That's a shame, because is energetic and inventive in bed. But, the peace deal gets cemented with his marriage to Levru from the Hiord clan. Technically, adultery is a crime, but you're sure you can persuade him to keep you as a lover. {| |- valign="top" |width="33%"|'''Heortling Heortling 2W''' * Heortling Customs * Heortling Myths * Housework * Sartar Geography * Weaving '''Steadwife 8W''' * Butchering * Domestic Chores * First Aid * Gardening * Husbandry * Maintain Peace in Stead * Manage Household * Preserve Food * Threshing * Weaving '''God-Talker 16W''' * Initiate to Ernalda * Perform Sacrifice * Pray to Ernalda * Sense Gods Nearby * Sense Spirits Nearby * Priest 5W * Chant for Hours * Dance Sacred Dances * Ernaldan Philosophy * Lead Worship |width="34%"|'''Devotee of Kadone (Ernalda the Queen) 9W''' * Ernalda Myths * Meaningful Glare 7WW * Orate * Persuade 17W * ''Earth Queen'' 4W (Banish Air Daimon, Command Earth Animal, Command Earth Daimon, Dismiss Air Elemental, Dismiss Earth Elemental, Exorcise Air Spirit Exorcise Earth Spirit, Enchant Copper ritual, Summon Talosi) * ''Find Another Way'' 11W (Calm Anger, Create Ring, Rally Women, Shame Husband, Stop Argument, Think of the Children) * ''Heaviness'' 12W (Force Flyer to Ground, Keep Object on Ground, Make Person Heavy, Make Object Heavy, Realization of Responsibility, Show Error of Ways, Stop Fleeing) |width="33%"|'''Relationships''' * Blue Spice Clan (birth clan) 4W * Clod the Talosi 2W (Large 2W, Strong 8W, Swallow Victim 4W) * Ex-wife of Mortality 18 * Lover of Askos 2W * Singing Rock Clan 12W * Singing Rock Inner Ring Member 14W * Singing Rock Loomhouse 2WW * Work with Estave 18 '''Personality''' * Doesn't Suffer Fools Gladly 11W * Just 3W * Loving 18 * Motherly 16 * Pious 14W * Pragmatic 19W * Strong-Willed 4WW * Unmerciful to Enemies 19 '''Others''' * Be Heard 19 * Cutting Insult 18W * Ignore Man 12W * Stand Ground 15W * Understand Consequences 11W * Work Together 6W '''3 Hero Points''' |} == Estave (Hiord Clan) == The feud between your two clans is just tragic. You'd just left the Singing Rock clan to marry Sarotar in the Hiord Clan when the feud started. The feud meant that you weren't able to go back to visit your family in the Singing Rock Clan and you were left to make your own way in the suddenly hostile Hiord clan. You decided to tough it out, but it has been your earnest wish for years to find some resolution to the feud. This ambition hasn't always been popular in the Hiord clan, especially with Sarotar, but you've kept at it. And now it seems that the end is in sight. You've found Levru, a good, sweet girl, and persuaded her to marry your nephew Askos. The marriage will be the final step in the reconcilliation between your two clans. Sarotar is a good and loving husband, but he just doesn't see the feud in the same way that you do. There's been a lot of bad things done by both clans in the feud, and Sarotar now thinks that the Singing Rock clan cannot be trusted. His attitude has caused a lot of ill feeling recently and you're now not sure if your marriage will survive the reconciliation. You're having second thoughts about whether it's really worth it. {| |- valign="top" |width="33%"|'''Heortling 18''' * Heortling Customs * Heortling Myths * Housework * Sartar Geography * Weaving '''Steadwife 4W''' * Butchering * Domestic Chores * First Aid * Gardening * Husbandry * Maintain Peace in Stead * Manage Household * Preserve Food * Threshing * Weaving '''Matchmaker 18W''' * Judge Compatibility * Judge Personality 4WW * Know Genealogy * Know Local Bloodlines * Negotiate Marriage * Pass Surreptitious Message * Sense Real Motive |width="34%"|'''God-Talker 4W''' * Initiate to Ernalda * Perform Sacrifice * Pray to Ernalda * Sense Gods Nearby * Sense Spirits Nearby '''Devotee of Vela (Ernalda Allmother) 13W''' * Animal Lore * Control Children * Domestic Chores * Ernalda Myths * Plant Lore * ''Bless Family'' 14W (Beautify Self, Comfort Sick Child, Conceal Family, Ease Fears, Make Husband Laugh) * ''Bless Mothers'' 9W (Aid Conception, Aid Contraception, Bless Birth, Bless Courtship, Bless Domestic Animal, Bless Pregnancy) * ''Matchmaking'' 16W (Ask Ancestors for Blessing, Assess Boy, Assess Girl, Calm Ardour, Ignite Passion, Reconcile Parents) |width="33%"|'''Relationships''' * Aunt of Askos 3W * Grateful Families 4W * Hiord Clan 7W * Hiord Clan Loomhouse 13W * Maternal to Levru 19 * Singing Rock Clan 18 * Wife of Sarotar 6WW * Work with Dorasar 2W '''Personality''' * Generous 16W * Loving 4WW * Motherly 17W * Pious 12W * Unmerciful to Enemies 18 '''Others''' * Calm Emotions 14W * Gossip 17W * Look Harmless 4W * Nag 4WW * Pass Unseen 18 * See the Best in People 9W '''3 Hero Points''' |} == Sarotar (Hiord Clan) == You’ve been defending the Hiord Clan against those bastards in the Singing Rock clan for years now. During that time, you’ve risen to a position of prominence in the clan. Your devotion to Rigsdal, the Night Watchman, and your dependability, have ensured that the clan has come to rely on you to keep them safe. You like being in that position, as it means that you’re listened to by the clan, and your opinions carry weight. All that is set to change. If the peace deal goes ahead, there will be peace between the clans. You won’t be needed to stand guard every night, and you’re opinions won’t be treated so seriously in the clan moots. You’ll be a no-one. The worst thing about it is that your wife, Estave, has been instrumental in setting up the deal, and she went ahead with it despite you commanding her not to. Without her work in making the first contacts and then setting up the marriage between Levru and Askos, the reconciliation would never be about to happen. And if she’s not around, the deal may not happen still. That’s why you contacted Gringle to arrange a way to stop things. That’s why you contacted the Gagarthi to kidnap Levru on her way to the Singing Rock tula. That’s why you’re considering doing something truly awful to Estave, if that’s what’s needed. {| |- valign="top" |width="33%"|'''Heortling 2W''' * Farming * Heortling Customs 18W * Heortling Myths '''Warrior 18W''' * Endurance * Guard Camp * Know Singing Rock Clan Tactics * Sartar Geography * Hide in Cover * Stay Awake * Spot Ambush 12W * Listen * Spear and Shield Fighting * Sword and Shield Fighting * Javelin Fighting * Recognize Fighting Style * Recognize Foe * Sprint * Scout * Fyrd Combat '''Weaponthane 11W''' * Ambush * Axe and Shield Fighting * Climb * Hide in Cover * Riding |width="34%"|'''God-Talker 7W''' * Initiate to Orlanth * Perform Sacrifice * Pray to Orlanth * Sense Gods Nearby * Sense Spirits Nearby '''Devotee of Rigsdal (Orlanth Allfather) 6W''' * Orlanth Myths * Sense Change in Weather * Endure Cold 12W * Keen Hearing 8W * Keen Vision * Alert 7WW * ''Allfather'' 6W (Lead by Example, Protect Home, Silencing Bellow, Summon Family) * ''Making'' 2W (Acclaim Chieftain, Bless New Building, Bless Tools, Oversee Marriage ritual, Work Hard) * ''Vigilance'' 8W (Hear Any Sound, Ignore Distraction, Night Vision, See Far, Sleep With One Eye Open) '''Relationships''' * Husband to Estave 2W * Hiord Clan 9W * Hiord Clan Warband 13W * Co-conspirator with Gringle 7W * Gagarthi Bandits 18 |width="33%"|'''Personality''' * Loving Husband 18 * Brave 16W * Boastful 15 * Determined 5W * Fatherly 18 * Hard-Working 7W * Responsible 18W * Reliable 7WW * Resolute 11W * Pious 17 * Desire Power 18W '''Others''' * Lie 14W * Be Heard in Moot 4W * Persuade Others 3W * Give Commands 2WW '''3 Hero Points''' |} == Character relationship map == [[Image:Peace-process-character-map.png|Peace Process characters relationships]] This is a simple relationship map showing the player heroes and significant NPCs in this episode [[Category:Scenarios]] [[Category:HeroQuest]] [[Category:Glorantha]] 0ef304fecc80c7c0ca9f41bd3d4c1758aad5ef35 1895 1891 2009-01-12T14:37:58Z Neil 2 Added links to PDFs wikitext text/x-wiki These are the pre-generated characters for [[The Peace Process]] episode for [[:Category:HeroQuest|HeroQuest]]. {|align="right" style="width:50%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Download formatted versions of the [[:Image:Peace-process-episode.pdf|scenario (PDF, 1.1Mb)]] and [[:Image:Peace-process-characters-ww.pdf|characters (PDF, 161Kb)]]. |} __TOC__ If there are only five players, make Gringle an NPC. If there are four players, drop Mortality as a PC. If there are three or fewer players, the scenario is unlikely to work as there will not be sufficient interaction between the remaining heroes. == Askos (Singing Rock Clan) == Oh, to be young, healthy, and in love! Life is good, and you intend to go out there, grab it by the horns, and squeeze every last experience from it. You worship, no, you live your life as, Destor the Adventurer. You are dynamic, proud, vigourous, active. You move as often and as quickly as the winds. No-one can make you do anything! You have mixed feelings about the peace deal. You’re disappointed that it’s on the rug, because the clashes with the Hiord clan are a great and dashing adventure. On the other hand, things are starting to get a bit boring as you’ve come up against all the Hiord warriors several times already. The peace deal will have some benefits, though. Your aunt Estave (now married into the Hiord clan) has arranged for the deal to be cemented by your marriage to Levru, and she’s gorgeous. Your current lover, Dorasa (ex-wife of Mortality the clan champion; he doesn’t seem to mind, though you were almost hoping he would) is getting a bit wrinkled now, though she’s great in bed. Also, peace with the Hiords will mean that you’re free to take up new challenges! Huzzah! {| |- valign="top" |width="33%"|'''Heortling 19''' * Farming * Heortling Customs * Heortling Myths * Sartar Geography '''Warrior 7W''' * Endurance * Fyrd Combat * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen 3WW * Recognize Fighting Style * Recognize Foe * Scout * Spear and Shield Fighting * Spot Ambush * Sprint 18W * Stay Awake 18 * Sword and Shield Fighting |width="34%"|'''Devotee of Destor (Orlanth Adventurous) 14W''' * Boast * Brew Woad * Dodge Attack * Jump 2WW * Orlanth Myths * Sense Change in Weather * Thunder Brother Myths * ''Combat'' 16W (Aid Throw With Wind, Armour of Woad, Enchant Silver ritual, Flickering Blade, Leaping Shield, Overbear Foe, Sword Help) * ''Movement'' 2WW (Burst of Speed, Leap Over Obstacle, Leap Through Window, Run On Mud, Run up Cliffs, Sunset Leap; Find a Way, Lift Object With Winds) * ''Wind'' 11W (Call Clouds, Call Wind, Drive Away Clouds, Hear Words on Wind, Summon Umbroli) '''Initiate of Niskis (Orlanth Adventurous) 18''' * Sensuality |width="33%"|'''Personality''' * Active 14W * Boastful 4W * Brave 2W * Easily Distracted 19 * Intensely Emotional 2W * Noble 19 * Proud 4W '''Relationships''' * Betrothed to Levru 16 * Lover of Dorasa 3W * Nephew of Estave 6W * Rival of Mortality 18 * Singing Rock Clan 19 * Singing Rock Warband 19 * Susurrate the Umbroli 19 (Strong 8W, Throw Debris 2W), living in a silver neck torc '''Other Abilities''' * Climb 5W * Neatly Trimmed Beard 16 * Quick Wits 18 * Shout “Huzzah!” 18 * Swing 17W * Twinkling Eye 18 * Whisper Sweet Nothings 2W * Winning Smile 19 '''3 Hero Points''' |} == Mortality (Singing Rock Clan) == You used to be Brol the Farmer, until you were caught up in the first raid by the Hiord Clan, and killed. Your wife, Dorasa, managed to beg a Chalana Arroy priestess to resurrect you, but she couldn’t bring you all back. Some essential part of you, your vitality, stayed on the Other Side. Life had no meaning, so you drifted to the worship of Humakt, Death, who gave you a course to steer by. You had your second funeral when you gave your new life to Humakt and left your old life behind. You became a warrior and eventually the clan’s Swordthane (champion), and fought in many battles, mostly against the Hiord clan.. You bear them no rancour. When your ex-wife took Askos as a lover, your were disturbed more than you should have been. You have no real opinion on the peace settlement. Your chieftain has given his backing to the deal, and you have sworn to obey the chieftain in all things. Despite that, you feel some unease that warriors on both sides have committed sins against the other, and no punishment will be imposed for them. You dislike Gringle, the fyrd leader. He is too passive, too conservative. That is not the way to win battles. But he spurns all your advice and your offers to assist in the fyrd’s training. {| |- valign="top" |width="33%"|'''Heortling 16''' * Farming * Heortling Customs * Heortling Myths '''Warrior 16W''' * Endurance * Fyrd Combat * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen * Recognize Fighting Style * Recognize Foe * Sartar Geography * Scout * Spear and Shield Fighting * Spot Ambush * Sprint * Stay Awake * Sword and Shield Fighting 8WW '''Weaponthane 3W''' * Ambush 3W [Forbidden by Geas] * Axe and Shield Fighting * Climb * Hide in Cover 19 * Riding |width="33%"|'''Champion 9W''' * Brawling 2W * Greatsword Fighting 4WW * Ignore Pain * Know Neighbouring Warriors * See Opponent’s Weakness * Single Sword Fighting 18W * Strong * Thrown Knife Fighting 8W * Tough '''Personality''' * Brave 17 * Determined 18 * Dispassionate 14W * Fearless 8W * Grim 8W * Professional 18 * Honourable 4W |width="34%"|'''Devotee of Makla Mann (Humakt) 4W''' * Craft Weapon * Dagger Fighting 12W * Fyrd Tactics * Humakt Myths * Recognize Lie * Sense Assassin * ''Death'' 19 (Bless Corpse, Death Song Berserk, Fight Undead, Lay Ghost, Visage of Fear) * ''Honour'' 8W (Empower Oath, Know Truth, Sense Ambush, Sever Relationship, Shame Coward) * ''Loyalty'' 6W (Fight to the Death, Inspire Loyalty, Sense Disloyalty, Stand Loyal) * [Makla Mann’s Blessing: Never participate in ambush.] * [Cannot receive healing from Chalana Arroy healers] '''Relationships''' * Ex-Husband of Dorasar 6W * Jealous of Askos 3W * Legally Dead 10WW * Loyal to Singing Rock Clan 8W * Rival of Gringle 2W * Singing Rock Clan 18 * Singing Rock Warband 2W '''3 Hero Points''' |} == Gringle (Singing Rock Clan) == You are the clan’s Shieldthane, one of the Four Fighting Storms. You lead the fyrd, the militia formed from all the clan’s fighting men. You’ve held this position for decades, and you know exactly what’s going on. As the fyrd leader, you’re responsible for turning the young hot-headed boys into reliable and steadfast fighting men. And you’re good at it. One thing that makes you good at it is that you’ve learnt that what doesn’t kill us makes us stronger. That’s why you don’t want the feud with the Hiord clan to end. If the feud ends, there will be no opportunity left for the youngsters to test themselves and become strong. It’s strong young men that we’ll all need for the struggle to kick out the Lunars, not the soft layabouts they’ll become without you. You’ve come to know and respect many of the Hiord weaponthanes, particularly Sarotar. He’s fed you details of what will be happening when the young bride Levru appears, and you’ve arranged for the Gagarthi to kidnap her. After that, your job is to keep the searchers in the wrong place until your accomplices can fatally sabotage the Issaries heroquest. This may not be obeying your chieftain, your chosen leader, but it’s for the good of everyone. The fact that continuing the feud will continue your position of influence is all rather incidental. You also don’t like Mortality, the clan champion. He keeps trying to meddle with the fyrd and their training, trying to get them to charge and attack more. You know that’s just going to get them killed. {| |- valign="top" |width="33%"|'''Heortling 2W''' * Farming * Heortling Customs * Heortling Myths '''Warrior 16W''' * Endurance * Fyrd Combat 4WW * Guard Camp * Hide in Cover * Javelin Fighting * Know Hiord Clan Tactics * Listen * Recognize Fighting Style * Recognize Foe * Sartar Geography * Scout * Singing Rock Geography 3WW * Spear and Shield Fighting * Sprint 2W * Stay Awake * Sword and Shield Fighting '''Weaponthane 5W''' * Ambush 16W * Axe and Shield Fighting * Climb 2W * Hide in Cover * Riding |width="33%"|'''Warband Leader 2WW''' * Businesslike Fighter * Fyrd Tactics 5WW * Shieldwall Fighting * Spot Ambush '''Initiate of Skarkval (Orlanth Allfather) 3W''' * Orlanth Myths * Sense Change in Weather * ''Allfather'' * ''Combat 8W'' * ''Making'' '''Common Magic 19''' * Dry Clothes, Heal Flesh, Hide in the Open, Keep Wind Out, Light Fire '''Personality''' * Brave 17 * Dedicated 19 * Dislike cocky young men 3W * Fatherly 6W * Hard-Working 2W * Professional 18W * Responsible 5W |width="34%"|'''Relationships''' * Co-conspirator with Sarotar 18W * Hate Hiord Clan 4W * Respect of Hiord Warband 4W * Rival of Mortality 18 * Singing Rock Clan 18W * Singing Rock Warband 3WW * Four Storms: :Olend (Swordthane) 19 (Warrior 3W, Finovan 19) :Sartark (Shieldthane) 12W (Warrior 4W, Starkval 19) :Govoran (Spearthane) 6W (Archer 2W, Helamakt 18) :Janerra (Backboy) 8W (Healer 12W, Bevara 8W) '''Others''' * Belittle Warrior 15W * Boast 6W * Displace Blame 4W * Gammy Leg 18 * Misdirect Question 7W * Seen it All Before 17W * Sharp Words 2W * Spread Rumours 2W '''3 Hero Points''' |} == Dorasa (Singing Rock Clan) == You represent Ernalda in the Singing Rock Clan's inner ring. As such, it's your job to make sure that the men-folk see what the sensible thing is and then do it, rather than swanning off on some other trivial but exciting distraction. In any situation, it's obvious to you what the right thing to do is. That's the easy part. What's hard is making everyone else see it, and then getting them to do it. Life would be so much easier if people just let you make the decisions, and they did what you told them. That way, you could have the whole clan pulling in the same direction and you could achieve so much more. The problem you've got at the moment is this peace deal with the Hiord clan. It's the right thing to do: that's obvious. That's why you've negotiated the deal in the first place. The cost in lives, cattle, limbs, and heartache is enormous and a real distraction from the real business of growing and thriving. The trouble is, many of the menfolk seem to enjoy the fighting and the raiding instead of staying at home ploughing and herding. You don't think that everyone is as supportive of your initiative as they should be. The only downside to the deal is that it means you should stop having sex with Askos, the youngster you took as a lover when your husband died and came back as Mortality the Humakti. That's a shame, because is energetic and inventive in bed. But, the peace deal gets cemented with his marriage to Levru from the Hiord clan. Technically, adultery is a crime, but you're sure you can persuade him to keep you as a lover. {| |- valign="top" |width="33%"|'''Heortling Heortling 2W''' * Heortling Customs * Heortling Myths * Housework * Sartar Geography * Weaving '''Steadwife 8W''' * Butchering * Domestic Chores * First Aid * Gardening * Husbandry * Maintain Peace in Stead * Manage Household * Preserve Food * Threshing * Weaving '''God-Talker 16W''' * Initiate to Ernalda * Perform Sacrifice * Pray to Ernalda * Sense Gods Nearby * Sense Spirits Nearby * Priest 5W * Chant for Hours * Dance Sacred Dances * Ernaldan Philosophy * Lead Worship |width="34%"|'''Devotee of Kadone (Ernalda the Queen) 9W''' * Ernalda Myths * Meaningful Glare 7WW * Orate * Persuade 17W * ''Earth Queen'' 4W (Banish Air Daimon, Command Earth Animal, Command Earth Daimon, Dismiss Air Elemental, Dismiss Earth Elemental, Exorcise Air Spirit Exorcise Earth Spirit, Enchant Copper ritual, Summon Talosi) * ''Find Another Way'' 11W (Calm Anger, Create Ring, Rally Women, Shame Husband, Stop Argument, Think of the Children) * ''Heaviness'' 12W (Force Flyer to Ground, Keep Object on Ground, Make Person Heavy, Make Object Heavy, Realization of Responsibility, Show Error of Ways, Stop Fleeing) |width="33%"|'''Relationships''' * Blue Spice Clan (birth clan) 4W * Clod the Talosi 2W (Large 2W, Strong 8W, Swallow Victim 4W) * Ex-wife of Mortality 18 * Lover of Askos 2W * Singing Rock Clan 12W * Singing Rock Inner Ring Member 14W * Singing Rock Loomhouse 2WW * Work with Estave 18 '''Personality''' * Doesn't Suffer Fools Gladly 11W * Just 3W * Loving 18 * Motherly 16 * Pious 14W * Pragmatic 19W * Strong-Willed 4WW * Unmerciful to Enemies 19 '''Others''' * Be Heard 19 * Cutting Insult 18W * Ignore Man 12W * Stand Ground 15W * Understand Consequences 11W * Work Together 6W '''3 Hero Points''' |} == Estave (Hiord Clan) == The feud between your two clans is just tragic. You'd just left the Singing Rock clan to marry Sarotar in the Hiord Clan when the feud started. The feud meant that you weren't able to go back to visit your family in the Singing Rock Clan and you were left to make your own way in the suddenly hostile Hiord clan. You decided to tough it out, but it has been your earnest wish for years to find some resolution to the feud. This ambition hasn't always been popular in the Hiord clan, especially with Sarotar, but you've kept at it. And now it seems that the end is in sight. You've found Levru, a good, sweet girl, and persuaded her to marry your nephew Askos. The marriage will be the final step in the reconcilliation between your two clans. Sarotar is a good and loving husband, but he just doesn't see the feud in the same way that you do. There's been a lot of bad things done by both clans in the feud, and Sarotar now thinks that the Singing Rock clan cannot be trusted. His attitude has caused a lot of ill feeling recently and you're now not sure if your marriage will survive the reconciliation. You're having second thoughts about whether it's really worth it. {| |- valign="top" |width="33%"|'''Heortling 18''' * Heortling Customs * Heortling Myths * Housework * Sartar Geography * Weaving '''Steadwife 4W''' * Butchering * Domestic Chores * First Aid * Gardening * Husbandry * Maintain Peace in Stead * Manage Household * Preserve Food * Threshing * Weaving '''Matchmaker 18W''' * Judge Compatibility * Judge Personality 4WW * Know Genealogy * Know Local Bloodlines * Negotiate Marriage * Pass Surreptitious Message * Sense Real Motive |width="34%"|'''God-Talker 4W''' * Initiate to Ernalda * Perform Sacrifice * Pray to Ernalda * Sense Gods Nearby * Sense Spirits Nearby '''Devotee of Vela (Ernalda Allmother) 13W''' * Animal Lore * Control Children * Domestic Chores * Ernalda Myths * Plant Lore * ''Bless Family'' 14W (Beautify Self, Comfort Sick Child, Conceal Family, Ease Fears, Make Husband Laugh) * ''Bless Mothers'' 9W (Aid Conception, Aid Contraception, Bless Birth, Bless Courtship, Bless Domestic Animal, Bless Pregnancy) * ''Matchmaking'' 16W (Ask Ancestors for Blessing, Assess Boy, Assess Girl, Calm Ardour, Ignite Passion, Reconcile Parents) |width="33%"|'''Relationships''' * Aunt of Askos 3W * Grateful Families 4W * Hiord Clan 7W * Hiord Clan Loomhouse 13W * Maternal to Levru 19 * Singing Rock Clan 18 * Wife of Sarotar 6WW * Work with Dorasar 2W '''Personality''' * Generous 16W * Loving 4WW * Motherly 17W * Pious 12W * Unmerciful to Enemies 18 '''Others''' * Calm Emotions 14W * Gossip 17W * Look Harmless 4W * Nag 4WW * Pass Unseen 18 * See the Best in People 9W '''3 Hero Points''' |} == Sarotar (Hiord Clan) == You’ve been defending the Hiord Clan against those bastards in the Singing Rock clan for years now. During that time, you’ve risen to a position of prominence in the clan. Your devotion to Rigsdal, the Night Watchman, and your dependability, have ensured that the clan has come to rely on you to keep them safe. You like being in that position, as it means that you’re listened to by the clan, and your opinions carry weight. All that is set to change. If the peace deal goes ahead, there will be peace between the clans. You won’t be needed to stand guard every night, and you’re opinions won’t be treated so seriously in the clan moots. You’ll be a no-one. The worst thing about it is that your wife, Estave, has been instrumental in setting up the deal, and she went ahead with it despite you commanding her not to. Without her work in making the first contacts and then setting up the marriage between Levru and Askos, the reconciliation would never be about to happen. And if she’s not around, the deal may not happen still. That’s why you contacted Gringle to arrange a way to stop things. That’s why you contacted the Gagarthi to kidnap Levru on her way to the Singing Rock tula. That’s why you’re considering doing something truly awful to Estave, if that’s what’s needed. {| |- valign="top" |width="33%"|'''Heortling 2W''' * Farming * Heortling Customs 18W * Heortling Myths '''Warrior 18W''' * Endurance * Guard Camp * Know Singing Rock Clan Tactics * Sartar Geography * Hide in Cover * Stay Awake * Spot Ambush 12W * Listen * Spear and Shield Fighting * Sword and Shield Fighting * Javelin Fighting * Recognize Fighting Style * Recognize Foe * Sprint * Scout * Fyrd Combat '''Weaponthane 11W''' * Ambush * Axe and Shield Fighting * Climb * Hide in Cover * Riding |width="34%"|'''God-Talker 7W''' * Initiate to Orlanth * Perform Sacrifice * Pray to Orlanth * Sense Gods Nearby * Sense Spirits Nearby '''Devotee of Rigsdal (Orlanth Allfather) 6W''' * Orlanth Myths * Sense Change in Weather * Endure Cold 12W * Keen Hearing 8W * Keen Vision * Alert 7WW * ''Allfather'' 6W (Lead by Example, Protect Home, Silencing Bellow, Summon Family) * ''Making'' 2W (Acclaim Chieftain, Bless New Building, Bless Tools, Oversee Marriage ritual, Work Hard) * ''Vigilance'' 8W (Hear Any Sound, Ignore Distraction, Night Vision, See Far, Sleep With One Eye Open) '''Relationships''' * Husband to Estave 2W * Hiord Clan 9W * Hiord Clan Warband 13W * Co-conspirator with Gringle 7W * Gagarthi Bandits 18 |width="33%"|'''Personality''' * Loving Husband 18 * Brave 16W * Boastful 15 * Determined 5W * Fatherly 18 * Hard-Working 7W * Responsible 18W * Reliable 7WW * Resolute 11W * Pious 17 * Desire Power 18W '''Others''' * Lie 14W * Be Heard in Moot 4W * Persuade Others 3W * Give Commands 2WW '''3 Hero Points''' |} == Character relationship map == [[Image:Peace-process-character-map.png|Peace Process characters relationships]] This is a simple relationship map showing the player heroes and significant NPCs in this episode [[Category:Scenarios]] [[Category:HeroQuest]] [[Category:Glorantha]] d91046dc98ec00baa0cc46f43790c9fe2971b311 14 1507 1888 1664 2009-01-08T22:11:17Z Neil 2 Added RPG category tag wikitext text/x-wiki Here are articles pertaining to the HeroQuest game system, as used for role-playing games set in [[:Category:Glorantha|Glorantha]]. <small>Content relating to Glorantha and/or HeroQuest is subject to [http://www.glorantha.com/inc/required_statements.html Issaries Inc's IP policy statements].</small> [[Category:RPGs]] 46dd3c091a2434d386684183adcddaecea6838ae 6 1560 1892 2009-01-12T14:28:58Z Neil 2 Formatted version of [[The Peace Process]] episode. [[Category:Scenarios]] wikitext text/x-wiki Formatted version of [[The Peace Process]] episode. [[Category:Scenarios]] 5b6b48cca82c0a024b6c2f1da9349f5a7c73c0ad 6 1561 1893 2009-01-12T14:29:54Z Neil 2 Formatted PDF of [[Peace Process characters|player characters]] for [[The Peace Process]] episode. [[Category:Scenarios]] wikitext text/x-wiki Formatted PDF of [[Peace Process characters|player characters]] for [[The Peace Process]] episode. [[Category:Scenarios]] 6008627e115353b19e7dbd18c92beaca82ed4e50 0 1526 1896 1868 2009-01-14T14:32:40Z Neil 2 /* Add KMenu actions */ Added VMWare server section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnugb * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] ba0f7980ace2f434781422c4daf90dc0a31e5f8a 1897 1896 2009-01-14T14:33:36Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 1c90bbae16c7cb2961a26d13f7c6a6d9ff0bfe50 1898 1897 2009-01-14T22:28:27Z Neil 2 /* Firefox extensions */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] a9f8c089cda1a9e6908e708ffe629813a1c119d3 1909 1898 2009-01-26T09:47:03Z Neil 2 /* Additional repositories */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] a56fc04bdda30173eabb088f623f44f9e92290fd 1910 1909 2009-01-26T09:55:03Z Neil 2 /* Additional repositories */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 9483d6f3f45a8f68de74a5f62dfea4c76d8c605e 1913 1910 2009-01-26T11:58:48Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Install the Sun Java using the apt package system. Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 72f14864c5c1796aefe2839af90c8fbc0225e700 1916 1913 2009-01-28T11:30:40Z Neil 2 /* Install and use Sun Java */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * kubuntu-restricted-extras (for MP3 and other playback) (it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 50c06830a97e999907eee5fbd55c9617b1454f9a 1930 1916 2009-02-02T14:22:19Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 3e0029f3e7edf4103dc5ba07d69fdaa676993890 0 1515 1901 1861 2009-01-19T12:26:53Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * ''[[Ruby on Rails setup]]'' | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] fc29d92e45ebcbb0e1de6907553be14e7557829a 1908 1901 2009-01-21T15:36:54Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * ''[[Ruby on Rails setup]]'' | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. [[Category:server setup]] 3821cf6c158ef2da7b89de89e03ed8054f2e6474 0 1563 1902 2009-01-19T12:40:15Z Neil 2 New page: Basic instructions: * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] * Install updated HPLIP (see [[C... wikitext text/x-wiki Basic instructions: * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] * Install updated HPLIP (see [[CUPS server setup|printer setup]] page). * Add xsane and sane-utils packages (sane-utils only for non-GUI machines) * Add to <code>/etc/udev/rules.d/45-libsane.rules</code> (base file available from [https://bugs.launchpad.net/ubuntu/+source/sane-backends/+bug/217571 this bug report]). # Hewlett-Packard Color LaserJet CM1312 MFP SYSFS{idVendor}=="03f0", SYSFS{idProduct}=="4e17", MODE="664", GROUP="scanner" * Restart <code>udev</code> * Reconnect device physically for rules change to take effect. (Reboot machine?) a730a8f5811d7c721a038c768055e180a950ac8c 1903 1902 2009-01-19T17:55:56Z Neil 2 wikitext text/x-wiki Basic instructions: * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] * Install updated HPLIP (see [[CUPS server setup|printer setup]] page). * Add xsane and sane-utils packages (sane-utils only for non-GUI machines) * Add to <code>/etc/udev/rules.d/45-libsane.rules</code> (base file available from [https://bugs.launchpad.net/ubuntu/+source/sane-backends/+bug/217571 this bug report]). # Hewlett-Packard Color LaserJet CM1312 MFP SYSFS{idVendor}=="03f0", SYSFS{idProduct}=="4e17", MODE="664", GROUP="scanner" * Add to <code>/etc/udev/rules.d/40-permissions.rules</code> # USB serial converters SUBSYSTEM=="usb_device", GOTO="usb_serial_start" SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", GOTO="usb_serial_start" GOTO="usb_serial_end" LABEL="usb_serial_start" ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", \ MODE="0660", GROUP="dialout" # Hewlett-Packard Color LaserJet CM1312 MFP ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", MODE="664", GROUP="scanner" LABEL="usb_serial_end" * Restart <code>udev</code> * Reconnect device physically for rules change to take effect. (Reboot machine?) cc9b14561a57522d4575d273db3ca5f1bd3231b4 1904 1903 2009-01-21T14:37:05Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} == Server setup == (We're using the desktop machine as the server for the scanning service.) * Install the updated HPLIP so that the scanner works (see [[CUPS server setup|printer setup]] page). * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# apt-get xsane sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# /etc/init.d/xinetd restart == Linux client setup == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# apt-get sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup == == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] b8ec5797857cbcd671abb75635e05fd43b886ac2 1905 1904 2009-01-21T15:33:00Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} == Server setup == (We're using the desktop machine as the server for the scanning service.) * Install the updated HPLIP so that the scanner works (see [[CUPS server setup|printer setup]] page). * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# apt-get xsane sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [http://ubuntuforums.org/showthread.php?p=4916614 this forum post] and [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# /etc/init.d/xinetd restart * As this is a multi-function printer, restart CUPS and check that it can still print to the printer. == Linux client setup == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# apt-get sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup == == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] daad951a45add59d0feb5b4ea8712de7d74e3e43 1912 1905 2009-01-26T10:41:14Z Neil 2 /* Server setup */ Added firewall rules wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} == Server setup == (We're using the desktop machine as the server for the scanning service.) * Install the updated HPLIP so that the scanner works (see [[CUPS server setup|printer setup]] page). * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# apt-get xsane sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [http://ubuntuforums.org/showthread.php?p=4916614 this forum post] and [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# /etc/init.d/xinetd restart * Open the right port in the [[Firewall setup|firewall]] by adding these lines to <code>/etc/iptables.rules</code>. Make sure the scanner is only available to machines on this LAN. ## Network scanning (port 6566) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport saned -j ACCEPT :and restart the firewall root@desktop:~# /etc/init.d/iptables restart * As this is a multi-function printer, restart CUPS and check that it can still print to the printer. == Linux client setup == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# apt-get sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup == == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] 6b0cea34f2899351550c37392f04ffc0918e224d 1917 1912 2009-01-28T20:36:04Z Neil 2 /* Server setup */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} == Server setup == (We're using the desktop machine as the server for the scanning service.) * Install the updated HPLIP so that the scanner works (see [[CUPS server setup|printer setup]] page). * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# apt-get xsane xsane-doc sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [http://ubuntuforums.org/showthread.php?p=4916614 this forum post] and [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# /etc/init.d/xinetd restart * Open the right port in the [[Firewall setup|firewall]] by adding these lines to <code>/etc/iptables.rules</code>. Make sure the scanner is only available to machines on this LAN. ## Network scanning (port 6566) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport saned -j ACCEPT :and restart the firewall root@desktop:~# /etc/init.d/iptables restart * As this is a multi-function printer, restart CUPS and check that it can still print to the printer. == Linux client setup == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# apt-get sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup == == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] 3ee750ba8492fb92fcdb9b55af2b9a3fe9ebf233 1918 1917 2009-01-28T20:41:48Z Neil 2 /* Windows client setup */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} == Server setup == (We're using the desktop machine as the server for the scanning service.) * Install the updated HPLIP so that the scanner works (see [[CUPS server setup|printer setup]] page). * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# apt-get xsane xsane-doc sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [http://ubuntuforums.org/showthread.php?p=4916614 this forum post] and [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# /etc/init.d/xinetd restart * Open the right port in the [[Firewall setup|firewall]] by adding these lines to <code>/etc/iptables.rules</code>. Make sure the scanner is only available to machines on this LAN. ## Network scanning (port 6566) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport saned -j ACCEPT :and restart the firewall root@desktop:~# /etc/init.d/iptables restart * As this is a multi-function printer, restart CUPS and check that it can still print to the printer. == Linux client setup == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# apt-get sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup == Use [http://sanetwain.ozuzo.net/ SaneTwain] and follow the instructions. Unfortunately, I couldn't get the scanner working reliably across a network at all, so SaneTwain didn't have a hope of working. == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] 00177f4090411fec7108d8378325264d1bf43716 0 1542 1906 1870 2009-01-21T15:35:08Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <tt>localhost</tt>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <tt>/etc/cups/cupsd.conf</tt> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock and add the <tt>Allow @LOCAL</tt> lines in the two <tt>Location</tt> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# /etc/init.d/cupsys restart You should then be able to point a webbrowser at <tt>desktop.server.tld:631</tt> or <tt>server.domain.tld:631</tt> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <tt>Administration</tt> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is only supported in HPLIP 2.8.10 and higher, but Ubuntu 8.04 only has HPLIP 2.8.2. So I need to update it. NB: Update the software before attaching the printer. * Remove the existing HPLIP installation. root@desktop:~# apt-get purge hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions] That's it! == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <tt><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></tt>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code>http://printers.domain.tld:631/printers</code>. Select the printer you want, such as <code>http://printers.domain.tld:631/printers/printer-raw</code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] 3c9a96b62128c7e3fb334fa17ad01a6e3f09c334 0 1538 1911 1850 2009-01-26T10:03:12Z Neil 2 /* Installation of Rails */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev ruby-extras :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev libruby-extras * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 462a22a4d802fd2666e12903b435ef7961b988a5 1914 1911 2009-01-26T13:33:00Z Neil 2 wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev ruby-extras :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev libruby-extras * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: 'project'@'localhost' password: === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold bd0afbac587dbb047b04a395df469a701eded1ee 1915 1914 2009-01-26T16:31:08Z Neil 2 /* Development and test databases */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev ruby-extras :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev libruby-extras * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.1.1.tgz root@server:~# rm rubygems-1.1.1.tgz root@server:~# cd rubygems-1.1.1 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.1.1 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 126d13bf628333c435c44782050cafa138d82d02 1931 1915 2009-02-02T15:55:17Z Neil 2 /* Installation of Rails */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold bc43bc7812b942bcfa9adbd875a2a59a4f460ad4 1932 1931 2009-02-02T15:59:28Z Neil 2 wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold c671d567dcb27dde79c050ecea9ff5b72bf6882a 1933 1932 2009-02-02T17:10:23Z Neil 2 /* Installation of Rails */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 1195740e5b5aedf83b6a0bbf98b01057258d2a48 1943 1933 2009-02-06T16:17:00Z Neil 2 /* Application deployment */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 6d27d7822de327844c4e3b85ce8c8620cb0b9e03 1944 1943 2009-02-06T16:22:01Z Neil 2 /* Subversion control setup */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which will be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold d26a040c6f870b73c2c3a764e1b86a2ed98728b2 1945 1944 2009-02-06T16:22:21Z Neil 2 /* Subversion control setup */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 0fa93ea9150a58e15f9a2907c2f6f0f9949717a6 1946 1945 2009-02-06T17:34:19Z Neil 2 /* Subversion control setup */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production .* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 5982c681f090e3a5ce63f06d7ccd8c42c462a085 1947 1946 2009-02-09T14:36:05Z Neil 2 /* Production database */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production.* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold f7575914cb5e201f37b129b317c0c51ffdbd245c 1948 1947 2009-02-09T17:30:40Z Neil 2 /* Application deployment */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production.* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files] * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 3931d4543580954ec678bb3b04918d471d6787cf 0 1535 1919 1769 2009-01-30T13:34:03Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} This one's easy. I use [http://www.squirrelmail.org/index.php SquirrelMail] as a webmail interface. To install Squirrelmail, install the packages: root@server:~# apt-get install squirrelmail ibritish (which also includes a British English dictionary). SquirrelMail has a very nice configuration script that does just about all the work for you, given that the [[Dovecot server setup|IMAP server]] and [[Web server setup|Web server]] are already working. Simply run the script: root@server:~# squirrelmail-configure select 'D' to configure using Dovecot, and follow the instructions. As SquirrelMail, Apache, and Dovecot are all running on the same machine, there's no need to use IMAPS to secure the communication between them. Next, you need to modify the <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> file to point to SquirrelMail. Adjust the <tt>DocumentRoot</tt> setting in the secure section of that file to DocumentRoot /usr/share/squirrelmail Restart the server: root@server:~# /etc/init.d/apache2 restart And you should have a SquirrelMail webmail site running! If you want to change the logo on the login screen, replace the file <tt>/usr/share/squirrelmail/images/sm_logo.png</tt> with another image, making sure it's the same width and height as the original. The site's favicon is <tt>/usr/share/squirrelmail/favicon.ico</tt>. To migrate from a previous installation, copy the files in <tt>/var/lib/squirrelmail/data</tt> to keep users' preferences. == See also == * [http://www.squirrelmail.org/documentation/ SquirrelMail documentation] * [https://help.ubuntu.com/community/Squirrelmail Hints for SquirrelMail on Ubuntu] [[Category:Server setup]] e3de966be16b68aa4b7a4fd1957c5b06d26fc2b5 6 1564 1920 2009-01-30T21:47:00Z Neil 2 A downloadable spreadsheet file that lists organisms from the [[:Category:Blue Planet|Blue Planet]] game against the biomes in which they occur. [[Category:Blue Planet]] wikitext text/x-wiki A downloadable spreadsheet file that lists organisms from the [[:Category:Blue Planet|Blue Planet]] game against the biomes in which they occur. [[Category:Blue Planet]] 484efce0f1e6f0b97b7eaeb95f223a4e4a5d0468 0 1477 1921 1615 2009-01-30T21:49:09Z Neil 2 Added link to CSV file wikitext text/x-wiki This is a list of what organisms are found in what biomes on Poseidon. You can also [[:Image:Biomes.csv.gz|download this as a compressed .csv file]]. === Key === MG = Moderators' Guide, NS = Natural Selection. For threat and resource levels, 0 = None, L = Low, M = Medium, H = High, VH = Very High. For abundance, 4 = Very Common, 3 = common, 2 = uncommon, 1 = rare. {| cellpadding=2 BORDER=1 |- ! colspan=5| &nbsp; ! colspan=18 align=center| Tropical ! colspan=13 align=center| Sub-tropical ! colspan=11 align=center| Temperate ! colspan=7 align=center| Polar |- ! Organism !! Reference !! Threat level !! Resource level !! Notes !! Rainforest !! Creeper forest !! Dry tropical forest !! Montane forest !! Tropical savanna !! Coast !! Tidal pool !! Estuary !! Marsh !! Bog !! Mangrove !! Canyonland !! Shallows and shelves !! Reef !! Seamount !! Open ocean surface !! Open ocean floor !! Sargassum island !! Montane forest !! Desert !! Coast !! Tidal pool !! Estuary !! Marsh !! Mangrove !! Shallows and shelves !! Reef !! Seamount !! Open ocean surface !! Open ocean floor !! Sargassum island !! Temperate forest !! Desert !! Coast !! Tidal pool !! Estuary !! Marsh !! Tidal mud reef !! Shallows and shelves !! Seamount !! Open ocean surface !! Open ocean floor !! Tundra !! Tidal pool !! Thermal Oasis !! Shallows and shelves !! Seamount !! Open ocean surface !! Open ocean floor |- | ''Am-Bush'' || MG 163 || M || M || Westscape and surroundings || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Angel Wings'' || NS 63 || 0 || 0 || Parasitic larvae || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; |- | ''Aurora Borialgae'' || NS 64 || L || ? || Makes up Skyscraper Reefs || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Bad Mojo'' || NS 65 || M || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Basilisk'' || NS 66 || 0 || M || &nbsp; || 3 || 3 || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Big Round Thing'' || NS 67 || L || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; |- | ''Blimp'' || MG 164 || M || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Blood Hunter'' || NS 69 || VH || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Bubble Array'' || NS 70 || L || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Carniflora'' || MG 165 || H || M || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Chain Beetle'' || MG 166 || H || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Chub'' || NS 71 || 0 || H || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Cup Sucker'' || NS 72 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Digger Crab'' || MG 167 || L || H || Mainly Haven || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Dune Creeper'' || NS 73 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Echo/Fish'' || NS 74 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Eel Dragon'' || MG 167 || 0 || L || Aircraft hazard || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Fast Fungus'' || MG 168 || L || 0 || &nbsp; || 3 || 3 || &nbsp; || 3 || 3 || 3 || 3 || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || 3 || &nbsp; || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Fish'' || MG 169 || Varies || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || 3 || 3 || 3 || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || 3 || &nbsp; || 3 || 3 || 3 || 3 |- | ''Fisherman'' || MG 170 || L || M || Haven Cluster || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Ghoster'' || MG 170 || H || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Gladiator Crab'' || NS 75 || M || H || Throughout Pacifica || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Glass Coral'' || NS 76 || M || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Greater White'' || MG 171 || VH || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 |- | ''Grendel'' || NS 78 || &nbsp; || &nbsp; || New Jamaica || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hangin' Joe'' || MG 172 || H || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hard Spurt'' || MG 187 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; |- | ''Harvester Worm'' || MG 173 || L || L || Long John slag, Pebble Rocks region || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hatchlings'' || MG 173 || H || 0 || Freshwater only || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hellbender'' || NS 79 || M || L || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Hexa Boar'' || MG 174 || M || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Howell's Leech'' || MG 175 || H || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Jellyroll'' || NS 80 || L || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || -3 || -3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || -3 || -3 || &nbsp; || &nbsp; || -2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Jump Jump'' || NS 81 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Keel Vine'' || NS 82 || M || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Land Lizard'' || MG 175 || H || M || &nbsp; || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Lesser White'' || NS 85 || H || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; |- | ''Leviathan'' || NS 8C || M || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; |- | ''Loggerhead'' || MG 176 || H || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Marine Iguana'' || &nbsp; || L || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Marsh Devil'' || MG 177 || H || 0 || Sierra Neuva || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Needle Bush'' || MG 177 || M || H || &nbsp; || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Needle Shell'' || MG 178 || M || M || Skyscraper reefs, Westscape || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Night Crawler'' || MG 179 || M || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Ni&ntilde;o Muerto'' || NS 86 || H || 0 || Zion, Channel, Prime Meridian || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Nooniebird'' || MG 179 || 0 || M || Westscape and surroundings || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Pharium'' || NS 87 || L || H || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Polypod'' || MG 180 || VH / 0 || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || 2 || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || 2 || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Kelp'' || MG 181 || L || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Mangrove'' || MG 182 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Potato'' || NS 88 || 0 || H || &nbsp; || 4 || 4 || 4 || 4 || 4 || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Sargassum'' || NS 89 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || 4 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Scorpion'' || MG 182 || VH || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Poseidon Trilobyte'' || MG 183 || L || H || New Hawaii || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Pump Weed'' || NS 91 || 0 || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Reefer Colony'' || NS 92 || 0 || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Reefworm'' || MG 184 || 0 || M || Northwest Territories || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Rubber Shrimp'' || MG 184 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; |- | ''Rumble Bee'' || MG 185 || 0 || H || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Saltwater Pseudoeel'' || NS 95 || H || L || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || 2 || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Sand Archer'' || NS 93 || M || L || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Schooler'' || MG 185 || 0 || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || 2 || 2 || 2 || 2 || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; |- | ''Sea Ghoul'' || NS 94 || 0 || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 4 || &nbsp; || 4 || 4 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 4 || 4 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 4 || 4 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 4 || 4 || 3 || &nbsp; || &nbsp; || &nbsp; |- | ''Seaweaver'' || MG 186 || ? || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Singer-in-the-Dark'' || NS 96 || H || ? || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Snow Weasel'' || NS 97 || M || ? || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; |- | ''Soft Spurt'' || MG 187 || H || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; |- | ''Squealers, Stick Monkeys'' || MG 188 || 0 || 0 || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Stone Snake'' || MG 188 || VH || 0 || Zion islands || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''String Worm'' || NS 98 || H || M || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Sunburst'' || MG 189 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; |- | ''Sweet Noodles'' || NS 99 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Thornrow'' || NS 100 || L || M || Westscape and surroundings || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Trident Fish'' || MG 190 || 0 || H || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; |- | ''Walkabout'' || MG 191 || H || ? || Prime Meridian || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Water Dart'' || MG 192 || H || 0 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || ? || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Water Hemp'' || MG 192 || 0 || H || &nbsp; || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Water Rat'' || MG 193 || 0 || 0 || Zion islands || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || 3 || 3 || 3 || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Weedeater'' || NS 101 || 0 || H || Kelp forests || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 3 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Wraparound'' || NS 103 || L || 0 || Fish parasite || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || 1 || 1 || &nbsp; || &nbsp; || 1 || 1 || 1 || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Xenosilicabenthoid'' || NS 104 || 0 || ? || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 1 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |- | ''Zipper'' || NS 104 || L || H || Zion islands, NW Territories || 2 || 2 || 2 || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || 2 || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; || &nbsp; |} === Other unspecified creatures === {| cellpadding=3 border=1 |- | ''Wash'' || NS 17 |- | ''Cuddlesloth'' || NS 18 |- | ''Alex's Ribbonsquid'' || NS 21 |- | ''Storm Children'' || NS 23 |- | ''Articulated Rook'' || NS 24 |- | ''Creeping Towel'' || NS 24 |} [[Category:Blue Planet]] 8828a0b47e6b74689b62859ed361637dbce7ce94 0 1565 1922 2009-01-30T22:13:26Z Neil 2 New page: <blockquote>A [[:Category:HeroQuest|HeroQuest]] scenario by Neil Smith</blockquote> <blockquote>Edited by Mark Galeotti</blockquote> <blockquote>This scenario is copyright © Neil Smith 20... wikitext text/x-wiki <blockquote>A [[:Category:HeroQuest|HeroQuest]] scenario by Neil Smith</blockquote> <blockquote>Edited by Mark Galeotti</blockquote> <blockquote>This scenario is copyright © Neil Smith 2003, 2004.</blockquote> {|align="right" style="width:50%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Download formatted versions of the [[:Image:Light-and-Death-episode.pdf|scenario (PDF)]] and [[:Image:Light-and-Death-characters-ww.pdf|characters (PDF)]]. |} Note that this scenario was written before the ''Imperial Lunar Handbook: Volume II'' was written. Some parts of the scenario, particularly details of Illumination, have been superceded by that book. But things are still close enough for a bit of a romp. == Rules and Benchmarks == This episode is designed to be played in a single session of perhaps 3-4 hours. Six pre-generated characters are provided [[Light and Death characters|in a separate page]]. The episode is based in the city of New Raibanth in the Lunar Heartlands. It shouldn't be too hard to transplant it to other Lunar cities, but some work will be needed. You will need a copy of the HeroQuest core rulebook. It would also be very helpful to have access to Imperial Lunar Handbook 1: The Lunar Empire, {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Sections shaded and enclosed with exclamation marks are notes for using the episode as a demonstration. ! |} This episode stresses investigation and clever solutions. Heroes' best abilities should be around 1W to 8W. Contests are described in a short form, as '''task (resistance)''': ''typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''realise that the city magistrates could become involved (14)''': ''Dara Happan Customs''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. == Overview == Manakenus the Architect is a prominent member of the Shining Column Ray League of Raibanth. He has recently been charged with overseeing the construction of a new bridge across the Oslir river to further unite the cities of Old and New Raibanth. However, the stress of this job has forced Manakenus to listen to the teachings of Akrash, a Lunar fringe cultist. Akrash is offering the prospect of releasing Manakenus from the stress he is under by leading him to a form of Illumination. Once Illuminated, Manakenus will be free from all the bonds that cause his mental grief. Akrash starts this process by killing Manakenus’s wife. The heroes investigate and initially follow the clues that point to a group of Old Raibanth protesters. However, when they find the protesters, the heroes discover new evidence that points to Akrash. When they try to find him, they encounter the Broken Knee gang who are eventually persuaded to reveal that Akrash and Manakenus, together with his baby daughter, have gone to the bridge. The heroes rush to the scene and hopefully prevent Manakenus killing his daughter to achieve Illumination. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Such weighty theological issues as the worship of the transcendental Lunar goddess Rashorana and Illumination will be explored in future Imperial Lunar Handbooks, and will not take the form described here. The ‘Illumination’ Akrash peddles is more like a charismatic’s con trick, and not genuine Lunar Illumination. Likewise, the magic powers he manifests are not the real ones which will be available to those who follow Rashorana (whom, some say, grants no magic at all!), but reflect some strange cult of which he has been a member. ! |} == Act 1: Introduction == === Scene 1: To the Bridge! === ''Key points: The heroes help Manakenus escape from his villa, through a ring of fanatics. '' The heroes are at the villa of Manakenus, the chief architect of the New Oslir Bridge, on the morning of the official consecration of the building work. The heroes have been charged, by their League's patriarch, to ensure that Manakenus gets to the ceremony safe and sound. Just as they are preparing to leave, a group of protesters appears, obviously trying to prevent Manakenus from attending the ceremony. The heroes have to get themselves, Manakenus, and an associate of his by the name of Akrash, out of the villa. Manakenus is scared of the protesters, and doesn't want to leave the villa. The heroes may be able to '''overcome his fears (17)''': ''Persuasive, Command civilian (-5)'', but if not, then Akrash manages to persuade him that his duty lies at the ceremony. "These yelmies," he says, "may worship the Light, but they do not See. Your bridge, by spreading the gentle glow of Rufelza into their city and their hearts, will Heal them of their closed-minded refusal to accept what is new and better. You cannot allow them to defeat you, by keeping you away from your rightful place." A few heavies from the Shining Column Ray League guard the villa, and will prevent any of the protesters breaking into the villa. They will not stray from the villa or its grounds. It is up to the heroes to get Manakenus to the bridge. They have some choices about how to do it. Fighting the protesters is possible, and the heroes will have the assistance of villa’s guards. However, the heroes may '''realise that the city magistrates could become involved (14)''': ''Dara Happan Customs'', and their levy-men would likely arrest whoever started the fight. The protesters could be persuaded to leave, or let Manakenus leave, but the protesters are determined to stay (resistance of 20). Finally, the heroes could sneak out the back way, if they can persuade Manakenus that a man of his station should stoop to such measures. The protesters follow Manakenus and the heroes when they leave. === Scene 2: The Consecration === ''Key points: The heroes watch the consecration ceremony. '' The heroes arrive at the site of the bridge, for the ceremony that celebrates the laying of the first stone of the roadway. Manakenus climbs the first column, where he joins various bigwigs from the Raibanth and Lunar governments. Akrash goes with him, but the heroes soon lose sight of him. Sacrifices are made to Yelm, Sedenya, and Hwarin Dalthippa (a Lunar goddess associated with engineering and construction) and they are asked to bless and protect the bridge. A crowd watches the consecration, with some heckling from scattered groups of protesters. There is little, if anything, for the heroes to do here, so this scene is best skipped over quickly. However, if you want to extend this section, then a few options would be for a protester to try to make trouble or for the heroes to spot a pick-pocketing in the crowd (do they leave Manakenus’s side and intervene?). === Scene 3 (Plot Turn 1): A Murder is Discovered === ''Key points: On returning to his villa, the heroes discover Manakenus's wife murdered. They vow, and are asked, to investigate. '' After the consecration, Manakenus sends a message to heroes that he will be staying at the bridge for some time, and that the heroes should return to his villa for the pre-arranged formal meal. When they return to the villa, they find the front gate ajar and unguarded. The watchman lies dead, half-concealed under a bush. Inside the house, the heroes can hear sobbing. There are bloodstained footprints in the hall, made by several feet, and they lead out, down the path to the gate. In the main reception room, Delra (Manakenus's wife) lies messily murdered, with blood and brains spattered across the walls. A blood-soaked cudgel, the murder weapon, lies nearby. In the kitchen, the cook has also been killed, stabbed and bludgeoned. The sobbing comes from a bedroom where Chalda, the family's nanny, has barricaded herself with Amena, Manakenus's child. She must be '''persuaded to come out (18)''': ''persuade, issue order (-8), intimidate (-15)'', but both she and the baby are unharmed. The villa is otherwise deserted. The heroes will need to '''calm Chalda down (14)''': ''Comfort, Persuade, Bargain (-5)''; if successful, she will gladly tell the heroes her story, through her sobs. After Manakenus and the heroes left, the protesters also left, shadowed by the guards. Chalda decided to take Amena out for a walk, and they were gone for about an hour. When they returned, she glimpsed someone in yellow, Yelmic robes darting out of the villa gate. She came into the villa, saw Delra dead, and then hid in the bedroom until the heroes arrived. She is of no further use to the heroes, and will gratefully be sent off to another house in the League. The heroes should now realise that it is their duty to find the perpetrators of this crime and bring them to justice. If they don't, other members of the League make it plain that doing so is their duty. == Act 2: Confrontation == === Scene 1: Finding the Suspects === ''Key points: The heroes track the protesters to their hideout. '' As the protesters are the obvious suspects, the heroes need to find them first. Asking passers-by in the street will lead nowhere: no one saw the murderers escape. The owner of a fried sausage stall will remember a group of solar protesters returning to the villa, but they left hurriedly almost immediately after. He didn't know where they were going, but did overhear someone shouting about returning to the docks. '''Asking around after the protesters (14)''': ''Know Raibanth, Relationship to League (-5), (searching the docks +5)'' will lead the heroes to a boarding house near the docks on the Oslir. The owner, a fat, drunken slob, will, if '''persuaded (10)''': ''persuade, fast talk, wealth'' confirm that the top floor of the five-story block has been booked by "a bunch of whinging, snobby types from over th' rivuh. They're upstairs now" === Scene 2: Struggle in the Sunlight === ''Key points: The heroes get to question the protesters. '' Assuming the heroes don't take too long, the protesters will not be in their rooms. Instead, the heroes will find them gathered in discussion on the flat roof of the boarding house. They are terrified that they will be accused of the murders at Manakenus's villa, and are on the verge of cutting and running back to Old Raibanth. They're nervous and jumpy, but they won't initiate hostilities. A couple of protesters are guarding the stairs to the roof and will try to prevent the heroes, or anyone else, getting onto the roof. The appearance of the heroes will cause consternation among them, increased when they realise that the heroes are from Manakenus's League. The heroes can persuade them to talk, beat them into submission, or employ some combination of the two (use an extended contest; see the protesters’ statistics at the end). The roof is mainly flat, with some trapdoors for access, a shed for tool storage, and various lumps, bumps, and low walls. If the protestors are defeated, they are prepared to tell their story. If things are going too well for the heroes, maybe the protesters are not as feeble as they appear, or perhaps their leader, Murhardavu, has some unexpected magical power? Maybe he can Manifest Yelmic Authority 155, forcing Dara Happan characters to kneel before him? Or perhaps the boarding house has other tenants who don’t take kindly to heavy-handed heroes storming in? === Scene 3 (midpoint): Whose Pendant is This? === ''Key points: The heroes learn that the protesters did not murder Manakenus's wife. They did pick up a talisman dropped by the murderer, though, and the heroes recognise it as belonging to Akrash. '' Once the situation on the rooftop has calmed, the protesters will tell their story. They followed the heroes to the consecration ceremony, but were moved on by guards there. They then decided to return to Manakenus's villa, intending to set up a protest for when he returned. However, when they arrived at the villa, they found the gate open and decided to go in. They found Manakenus's wife already dead, and fled. One of their number picked up a blood-spattered silver pendant marked with Lunar symbols lying near the wife's body. On examining the pendant, they should recognise it as Akrash's (16): Keen eye, Sedenyic philosophy. On listening to the protesters’ story, the heroes should '''realise that their story is true (14)''': ''Detect lies, Discern truth, Persuade (-5)''. If the heroes were defeated by the solars, they are forced to retreat, but notice (and perhaps purloin) Akrash's pendant lying on the floor. === Scene 3a: Meanwhile... === ''Key points: What Akrash and Manakenus are doing. '' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Note: this “scene” is for Narrator information only, not for the heroes to play through. You may want to use it as a “cut scene” if you and the players would enjoy it. ! |} While the heroes are tracking down the protesters, Akrash and Manakenus are not idle. Manakenus stays at the consecration ceremony for a while, until he hears of the murder of his wife. He then rushes home to see her, and is taken to where his baby and nanny are staying. He stays with them for a few hours, until Akrash turns up to offer him comfort and support. While they are alone, Akrash persuades Manakenus that he will only be free of his pain by severing his remaining ties and becoming Illuminated. Manakenus agrees, and takes his baby to Akrash's church where they pray for a while. After nightfall they, together with some Broken Knee heavies, go to the bridge site for the final confrontation. === Scene 4 (Plot Turn 2): At the Church === ''Key points: The heroes rush to Akrash's church, where they are confronted by a number of the Broken Knee gang. They learn that Akrash, Manakenus, and his baby daughter have left for a special ceremony on the new bridge.'' If the protesters are not to blame for the murders, who is? The trail points to Akrash, so the heroes will have to locate him. If they check, he is not at the villa or with Manakenus's nanny, though Chalda will say that Akrash and a distraught Manakenus left with the baby "to liberate themselves from grief" a few hours ago. She thinks they were going to Akrash's church, but they did also mention the bridge as the cause of all Manakenus's grief. The heroes may '''know the location of Akrash's church (18)''': ''Know Raibanth, Sedenyic Philosophy (-5), Initiate of Whoever (-12)''. Some of Manakenus's servants know where it is, or they can '''ask around in the city (18)''': ''Dara Happan customs, Underworld contacts (-5)''. The church is a converted shop in a residential area of the city near the docks. Outside, it is painted with Lunar symbols. When the heroes arrive, Akrash and Manakenus have already left, but three Broken Knee gang members are hanging around outside, guarding it. They'll try to scare off the heroes, and won't be too averse to using force if it's needed. If they look like they're losing the encounter, they will retreat into the church and lock the doors. The heroes may need to either break or sneak in finally to defeat the guards. When the guards are finally defeated and '''questioned (18)''': ''Discern truth, Persuade (-3)'', they will grudgingly confirm that Akrash, "some snotty Tenth [minor aristocrat] and a babe-in-arms" were at the church and prayed for a while, and then left for the site of the new bridge to perform some ceremony. The rest of the gang went with them. == Act 3: Resolution == === Scene 1: Going Up and Down === ''Key points: The heroes must fight their way up the scaffolding, through the rest of the Broken Knee gang, to prevent Akrash's nefarious plot. They are either helped or hindered by the remains of the protesters. '' It is now (or should be) night. The heroes rush to the site of the new bridge, and see a number of people atop the first tower, including Akrash and Manakenus. Manakenus is holding his baby. The rest of the people are members of the Broken Knee gang. Many are on the tower; some are on the scaffolding, and one is at the base of the tower. There is some religious equipment on the top of the tower too, different from what the heroes saw at the consecration ceremony. There are no guards or watchmen patrolling the fence. When the heroes arrive, Akrash is persuading Manakenus that his path to salvation and inner peace is through Illumination, and that this can be achieved by severing his final tie to mundane things: he must throw his baby daughter from the tower. Unless the heroes stop them, this is what happens. As the heroes rush up the scaffolding to Manakenus, the Broken Knee gangers rush down and a fight ensues on the scaffolding. There are about as many gang members as heroes, but the gang starts with 17AP for each hero; AP losses can be represented by gangers falling from the scaffolding. If the fight goes badly for the gang, Akrash will use his Lunar magic to lend them APs. If it's needed, the protesters appear at the bridge during the fight. How they were treated earlier by the heroes will determine how much assistance they lend them now, but the protesters are eager to stop Akrash and Manakenus, as they think that the ceremony has something to do with completing the bridge. Once the gang has been defeated, the heroes enter a contest with Akrash over the fate of Manakenus's baby. He wants Manakenus to throw down the baby; the heroes, presumably, don't. If the heroes succeed, Manakenus sees the error of his ways and breaks down in tears while Akrash flees. If the heroes fail, Manakenus throws his child to her death (unless the heroes can somehow save her) and then collapses in a state of religious rapture. === Scene 2 (Epilogue): Cut Scene: Akrash's reward === ''Key points: Akrash receives his just rewards. '' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! After the players have congratulated or consoled each other, the scene cuts to Akrash a few days later (the heroes are not present). ! |} He is in a plain office in the Mayor's palace, being interviewed by a Lunar minister about the events surrounding Manakenus and the bridge. If the heroes failed and Akrash persuaded Manakenus to sacrifice his child, the minister praises Akrash for a job well done and comments that the bridge now has a great champion in Manakenus, the flawed Illuminate; he will commit himself, body and soul, to completing the bridge. Akrash is presented with the Order of the Glory of the Embracing Moon, First Class. Five years later, the bridge is completed and Raibanth becomes united once more. If, on the other hand, Akrash was foiled, the minister he is seen berating him and blaming him for the probable failure of the bridge. The minister wonders if Akrash's talents would better serve the Empire with thirty years of hard labour in a Danfive Xaroni gulag in the Redlands. Akrash is never seen again. In Raibanth, the work on this bridge, like so many others, peters out after less than three years of turmoil and strife and it remains forever barely started. == Locations == === Raibanth === Glorious Raibanth is the ancient Solar capital of Dara Happa, predating the rise of the Red Moon, the incorporation of the Dara Happans in the Lunar Empire, and the creation of a new capital at Glamour. It is a proud and ancient city, with a population of some 75,000 souls, dominated by the huge gilded statue of Raiba, its mythical founder, and the ziggurat of the Overseer. The city is circular, following the orderly patterns laid down by those who follow Yelm, the Imperial Sun, but it is divided into three parts by the Oslir and Joat Rivers. New and Old Raibanth, on the western and eastern banks of the Oslir respectively, are dominated by the Yelmics, although there is a growing body of Lunar worshippers in New Raibanth in particular. They are joined by just the Bridge of Seven Saints, and although this is a huge span, across which six ox-carts can rumble side-by-side, this nonetheless limits the contact between New and Old Raibanth. Ever since the destruction of the Bridge of Broken Angels’ Wings shortly before the Birth of the Goddess, no bridges connect the main city to Amoli, on the other side of the Joat. This is likely to remain the case for some time, as Amoli is largely populated by Darjiini, resentful of their Dara Happan masters and determined to retain as much independence as they can. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Dara Happa is not Rome, nor is it Babylonia, nor is it Byzantium, although it displays some of the traits of all three. Especially for the purpose of a demonstration game, where players may well know nothing about Glorantha, it may be easiest to treat the city as being quite Roman in feel, although it is worth noting that this is an over-simplification. ! |} === The Bridge === This is the latest of several projects to span the Oslir river and reunite New and Old Raibanth; the latter is a hard-line Solar community which resists integration into the Lunar Empire. To the Lunar government, the bridge is a symbol of healing and inclusion in the Empire. The bridge will span the mighty Oslir in several arches. At the moment, construction on the bridge has barely started. The first of many support columns has been built 15 metres high on the river's edge. The first few stones of the roadway are placed atop this column in the ceremony of Act 1 Scene 2. The column remains covered in scaffolding and the area surrounding it is a building site, littered with stone blocks in various states of completion, piles of sand and gravel, various tools (some even locked in the numerous toolsheds). The site is surrounded by a fence and normally patrolled by private watchmen. On the scaffolding itself, there is all the paraphernalia of building work, just waiting to be used by or against the Broken Knee gang members. Hammers, chisels, ropes, buckets (some full of various liquids and powders), piles of stones and mortar, and the like all lie around the many levels of the scaffolding. Some of the flooring planks and the ladders are loose, ready to trip the unwary, and many more can be loosened quickly. Ropes slung over pulleys are everywhere, some attached to heavy loads (around the weight of a person), and all ready to be freed in an instant. In other words, there should be plenty of props to help you come up with an entertaining fight on the bridge column. === Manakenus's villa === Manakenus lives in a middling-sized villa in a well-to-do part of New Raibanth, as befits a man of his position. A high wall surrounds a spacious formal garden. The main double gate is on the main road; a smaller, single door opens out onto an alley at the rear. The villa is a circular, single story ring of rooms surrounding a central courtyard. In the courtyard is a small obelisk, a shrine to Yelm. The villa has numerous rooms. The reception rooms are at the front of the house, and large doors that open onto the courtyard. Private rooms are on the sides. Servants’ quarters, kitchens, and storerooms are at the back. Inside, the floors are tiled and the walls are wood panelled and covered in hanging rugs. Various objets d'art adorn the villa, hanging on the walls, in cabinets, or on plinths. == Dramatis Personae == === Manakenus === ''Age 34, male, architect, initiate of Yelm the Overseer and Hwarin Dalthippa the Builder. '' Manakenus is a prominent Tenth in the Shining Ray Column League (the same league as the heroes). He has recently been made Overseer of the construction of the new bridge, a high honour for one so young. However, the appointment has weighed heavy on his mind and he has begun to crack under the strain. He has come to rely on Akrash, and his eccentric brand of philosophy, for support through these times. He has a wife (Delra) and four-month-old daughter (Amena); Chaldra, a nanny, looks after Amena. Main Abilities: Civil engineer 17WWW, High class 2W, Oversee workers 3WW, Overseer magic 12W, Sedenyic philosophy 8W. === Akrash === ''Age 68, male, heretical Lunar holy man, devotee of Rashorana and many other aspects of the Red Moon, Illuminated (according to him). '' Akrash is an independent seeker after the Truth that lies behind the Masks of the Red Moon. He always wears a half-mask himself. His theories on Rashorana, Illumination, and other matters are out of line with mainstream Lunar practices, but he continues to practice and preach his version of the religion from his small church. He has recently become a great friend and confident of Manakenus and is with him a great deal. He is convinced that Manakenus's salvation lies in his own brand of ‘Illumination’ – entailing sacrificing all that matters to him in the mundane world that he can concentrate exclusively on Higher Truths. He is also an operative of the Spoken Word, one of the Instrumentalities (Lunar secret police agencies), or rather a dupe – he is under such deep cover that even he's convinced that he's a free agent. Main Abilities: Pious 4WW, Devoted to Rashorana 12WW, Own Brand of Sedenyic Philosophy 3WW, Persuasive 17W, Forceful personality 10W. Healing magics 2WW, Madness magics 18. === The Broken Knee gang === ''Small-time roughs '' The Broken Knee is a small gang of street heavies. Their territory includes Akrash's church, and he is paying them protection money, and a bit extra to act as his muscle. Typical gang member: Cudgel combat 17 (+3), Avoid Clever Stratagem 10, Resist Magic 14, Athletic 16, Relationship to Akrash 15. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Gang members should use their Avoid Clever Stratagem ability as their resistance to being tricked our outwitted by the heroes. ! |} === The Solar Protesters === ''Old Yelmic protesters. '' This is a small group of protesters from Old Raibanth. They have come over the Oslir to protest against, and hopefully stop, the construction of the new bridge. They are not violent, but will defend themselves. They are led by Murhardavu, an ageing Yelmic patriarch. They should be slightly comical, although possibly useful in the closing fight. Typical protester: Cudgel Combat 15 (small club +2), Chant Long-Winded Protest 2W, Conservative 18, Naïve 16. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] d1e20c3171bb5cd73586a807e0051919e123ff44 1923 1922 2009-01-30T22:14:10Z Neil 2 wikitext text/x-wiki {|align="right" style="width:50%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Download formatted versions of the [[:Image:Light-and-Death-episode.pdf|scenario (PDF)]] and [[:Image:Light-and-Death-characters-ww.pdf|characters (PDF)]]. |} <blockquote>A [[:Category:HeroQuest|HeroQuest]] scenario by Neil Smith</blockquote> <blockquote>Edited by Mark Galeotti</blockquote> <blockquote>This scenario is copyright © Neil Smith 2003, 2004.</blockquote> Note that this scenario was written before the ''Imperial Lunar Handbook: Volume II'' was written. Some parts of the scenario, particularly details of Illumination, have been superceded by that book. But things are still close enough for a bit of a romp. == Rules and Benchmarks == This episode is designed to be played in a single session of perhaps 3-4 hours. Six pre-generated characters are provided [[Light and Death characters|in a separate page]]. The episode is based in the city of New Raibanth in the Lunar Heartlands. It shouldn't be too hard to transplant it to other Lunar cities, but some work will be needed. You will need a copy of the HeroQuest core rulebook. It would also be very helpful to have access to Imperial Lunar Handbook 1: The Lunar Empire, {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Sections shaded and enclosed with exclamation marks are notes for using the episode as a demonstration. ! |} This episode stresses investigation and clever solutions. Heroes' best abilities should be around 1W to 8W. Contests are described in a short form, as '''task (resistance)''': ''typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''realise that the city magistrates could become involved (14)''': ''Dara Happan Customs''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. == Overview == Manakenus the Architect is a prominent member of the Shining Column Ray League of Raibanth. He has recently been charged with overseeing the construction of a new bridge across the Oslir river to further unite the cities of Old and New Raibanth. However, the stress of this job has forced Manakenus to listen to the teachings of Akrash, a Lunar fringe cultist. Akrash is offering the prospect of releasing Manakenus from the stress he is under by leading him to a form of Illumination. Once Illuminated, Manakenus will be free from all the bonds that cause his mental grief. Akrash starts this process by killing Manakenus’s wife. The heroes investigate and initially follow the clues that point to a group of Old Raibanth protesters. However, when they find the protesters, the heroes discover new evidence that points to Akrash. When they try to find him, they encounter the Broken Knee gang who are eventually persuaded to reveal that Akrash and Manakenus, together with his baby daughter, have gone to the bridge. The heroes rush to the scene and hopefully prevent Manakenus killing his daughter to achieve Illumination. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Such weighty theological issues as the worship of the transcendental Lunar goddess Rashorana and Illumination will be explored in future Imperial Lunar Handbooks, and will not take the form described here. The ‘Illumination’ Akrash peddles is more like a charismatic’s con trick, and not genuine Lunar Illumination. Likewise, the magic powers he manifests are not the real ones which will be available to those who follow Rashorana (whom, some say, grants no magic at all!), but reflect some strange cult of which he has been a member. ! |} == Act 1: Introduction == === Scene 1: To the Bridge! === ''Key points: The heroes help Manakenus escape from his villa, through a ring of fanatics. '' The heroes are at the villa of Manakenus, the chief architect of the New Oslir Bridge, on the morning of the official consecration of the building work. The heroes have been charged, by their League's patriarch, to ensure that Manakenus gets to the ceremony safe and sound. Just as they are preparing to leave, a group of protesters appears, obviously trying to prevent Manakenus from attending the ceremony. The heroes have to get themselves, Manakenus, and an associate of his by the name of Akrash, out of the villa. Manakenus is scared of the protesters, and doesn't want to leave the villa. The heroes may be able to '''overcome his fears (17)''': ''Persuasive, Command civilian (-5)'', but if not, then Akrash manages to persuade him that his duty lies at the ceremony. "These yelmies," he says, "may worship the Light, but they do not See. Your bridge, by spreading the gentle glow of Rufelza into their city and their hearts, will Heal them of their closed-minded refusal to accept what is new and better. You cannot allow them to defeat you, by keeping you away from your rightful place." A few heavies from the Shining Column Ray League guard the villa, and will prevent any of the protesters breaking into the villa. They will not stray from the villa or its grounds. It is up to the heroes to get Manakenus to the bridge. They have some choices about how to do it. Fighting the protesters is possible, and the heroes will have the assistance of villa’s guards. However, the heroes may '''realise that the city magistrates could become involved (14)''': ''Dara Happan Customs'', and their levy-men would likely arrest whoever started the fight. The protesters could be persuaded to leave, or let Manakenus leave, but the protesters are determined to stay (resistance of 20). Finally, the heroes could sneak out the back way, if they can persuade Manakenus that a man of his station should stoop to such measures. The protesters follow Manakenus and the heroes when they leave. === Scene 2: The Consecration === ''Key points: The heroes watch the consecration ceremony. '' The heroes arrive at the site of the bridge, for the ceremony that celebrates the laying of the first stone of the roadway. Manakenus climbs the first column, where he joins various bigwigs from the Raibanth and Lunar governments. Akrash goes with him, but the heroes soon lose sight of him. Sacrifices are made to Yelm, Sedenya, and Hwarin Dalthippa (a Lunar goddess associated with engineering and construction) and they are asked to bless and protect the bridge. A crowd watches the consecration, with some heckling from scattered groups of protesters. There is little, if anything, for the heroes to do here, so this scene is best skipped over quickly. However, if you want to extend this section, then a few options would be for a protester to try to make trouble or for the heroes to spot a pick-pocketing in the crowd (do they leave Manakenus’s side and intervene?). === Scene 3 (Plot Turn 1): A Murder is Discovered === ''Key points: On returning to his villa, the heroes discover Manakenus's wife murdered. They vow, and are asked, to investigate. '' After the consecration, Manakenus sends a message to heroes that he will be staying at the bridge for some time, and that the heroes should return to his villa for the pre-arranged formal meal. When they return to the villa, they find the front gate ajar and unguarded. The watchman lies dead, half-concealed under a bush. Inside the house, the heroes can hear sobbing. There are bloodstained footprints in the hall, made by several feet, and they lead out, down the path to the gate. In the main reception room, Delra (Manakenus's wife) lies messily murdered, with blood and brains spattered across the walls. A blood-soaked cudgel, the murder weapon, lies nearby. In the kitchen, the cook has also been killed, stabbed and bludgeoned. The sobbing comes from a bedroom where Chalda, the family's nanny, has barricaded herself with Amena, Manakenus's child. She must be '''persuaded to come out (18)''': ''persuade, issue order (-8), intimidate (-15)'', but both she and the baby are unharmed. The villa is otherwise deserted. The heroes will need to '''calm Chalda down (14)''': ''Comfort, Persuade, Bargain (-5)''; if successful, she will gladly tell the heroes her story, through her sobs. After Manakenus and the heroes left, the protesters also left, shadowed by the guards. Chalda decided to take Amena out for a walk, and they were gone for about an hour. When they returned, she glimpsed someone in yellow, Yelmic robes darting out of the villa gate. She came into the villa, saw Delra dead, and then hid in the bedroom until the heroes arrived. She is of no further use to the heroes, and will gratefully be sent off to another house in the League. The heroes should now realise that it is their duty to find the perpetrators of this crime and bring them to justice. If they don't, other members of the League make it plain that doing so is their duty. == Act 2: Confrontation == === Scene 1: Finding the Suspects === ''Key points: The heroes track the protesters to their hideout. '' As the protesters are the obvious suspects, the heroes need to find them first. Asking passers-by in the street will lead nowhere: no one saw the murderers escape. The owner of a fried sausage stall will remember a group of solar protesters returning to the villa, but they left hurriedly almost immediately after. He didn't know where they were going, but did overhear someone shouting about returning to the docks. '''Asking around after the protesters (14)''': ''Know Raibanth, Relationship to League (-5), (searching the docks +5)'' will lead the heroes to a boarding house near the docks on the Oslir. The owner, a fat, drunken slob, will, if '''persuaded (10)''': ''persuade, fast talk, wealth'' confirm that the top floor of the five-story block has been booked by "a bunch of whinging, snobby types from over th' rivuh. They're upstairs now" === Scene 2: Struggle in the Sunlight === ''Key points: The heroes get to question the protesters. '' Assuming the heroes don't take too long, the protesters will not be in their rooms. Instead, the heroes will find them gathered in discussion on the flat roof of the boarding house. They are terrified that they will be accused of the murders at Manakenus's villa, and are on the verge of cutting and running back to Old Raibanth. They're nervous and jumpy, but they won't initiate hostilities. A couple of protesters are guarding the stairs to the roof and will try to prevent the heroes, or anyone else, getting onto the roof. The appearance of the heroes will cause consternation among them, increased when they realise that the heroes are from Manakenus's League. The heroes can persuade them to talk, beat them into submission, or employ some combination of the two (use an extended contest; see the protesters’ statistics at the end). The roof is mainly flat, with some trapdoors for access, a shed for tool storage, and various lumps, bumps, and low walls. If the protestors are defeated, they are prepared to tell their story. If things are going too well for the heroes, maybe the protesters are not as feeble as they appear, or perhaps their leader, Murhardavu, has some unexpected magical power? Maybe he can Manifest Yelmic Authority 155, forcing Dara Happan characters to kneel before him? Or perhaps the boarding house has other tenants who don’t take kindly to heavy-handed heroes storming in? === Scene 3 (midpoint): Whose Pendant is This? === ''Key points: The heroes learn that the protesters did not murder Manakenus's wife. They did pick up a talisman dropped by the murderer, though, and the heroes recognise it as belonging to Akrash. '' Once the situation on the rooftop has calmed, the protesters will tell their story. They followed the heroes to the consecration ceremony, but were moved on by guards there. They then decided to return to Manakenus's villa, intending to set up a protest for when he returned. However, when they arrived at the villa, they found the gate open and decided to go in. They found Manakenus's wife already dead, and fled. One of their number picked up a blood-spattered silver pendant marked with Lunar symbols lying near the wife's body. On examining the pendant, they should recognise it as Akrash's (16): Keen eye, Sedenyic philosophy. On listening to the protesters’ story, the heroes should '''realise that their story is true (14)''': ''Detect lies, Discern truth, Persuade (-5)''. If the heroes were defeated by the solars, they are forced to retreat, but notice (and perhaps purloin) Akrash's pendant lying on the floor. === Scene 3a: Meanwhile... === ''Key points: What Akrash and Manakenus are doing. '' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Note: this “scene” is for Narrator information only, not for the heroes to play through. You may want to use it as a “cut scene” if you and the players would enjoy it. ! |} While the heroes are tracking down the protesters, Akrash and Manakenus are not idle. Manakenus stays at the consecration ceremony for a while, until he hears of the murder of his wife. He then rushes home to see her, and is taken to where his baby and nanny are staying. He stays with them for a few hours, until Akrash turns up to offer him comfort and support. While they are alone, Akrash persuades Manakenus that he will only be free of his pain by severing his remaining ties and becoming Illuminated. Manakenus agrees, and takes his baby to Akrash's church where they pray for a while. After nightfall they, together with some Broken Knee heavies, go to the bridge site for the final confrontation. === Scene 4 (Plot Turn 2): At the Church === ''Key points: The heroes rush to Akrash's church, where they are confronted by a number of the Broken Knee gang. They learn that Akrash, Manakenus, and his baby daughter have left for a special ceremony on the new bridge.'' If the protesters are not to blame for the murders, who is? The trail points to Akrash, so the heroes will have to locate him. If they check, he is not at the villa or with Manakenus's nanny, though Chalda will say that Akrash and a distraught Manakenus left with the baby "to liberate themselves from grief" a few hours ago. She thinks they were going to Akrash's church, but they did also mention the bridge as the cause of all Manakenus's grief. The heroes may '''know the location of Akrash's church (18)''': ''Know Raibanth, Sedenyic Philosophy (-5), Initiate of Whoever (-12)''. Some of Manakenus's servants know where it is, or they can '''ask around in the city (18)''': ''Dara Happan customs, Underworld contacts (-5)''. The church is a converted shop in a residential area of the city near the docks. Outside, it is painted with Lunar symbols. When the heroes arrive, Akrash and Manakenus have already left, but three Broken Knee gang members are hanging around outside, guarding it. They'll try to scare off the heroes, and won't be too averse to using force if it's needed. If they look like they're losing the encounter, they will retreat into the church and lock the doors. The heroes may need to either break or sneak in finally to defeat the guards. When the guards are finally defeated and '''questioned (18)''': ''Discern truth, Persuade (-3)'', they will grudgingly confirm that Akrash, "some snotty Tenth [minor aristocrat] and a babe-in-arms" were at the church and prayed for a while, and then left for the site of the new bridge to perform some ceremony. The rest of the gang went with them. == Act 3: Resolution == === Scene 1: Going Up and Down === ''Key points: The heroes must fight their way up the scaffolding, through the rest of the Broken Knee gang, to prevent Akrash's nefarious plot. They are either helped or hindered by the remains of the protesters. '' It is now (or should be) night. The heroes rush to the site of the new bridge, and see a number of people atop the first tower, including Akrash and Manakenus. Manakenus is holding his baby. The rest of the people are members of the Broken Knee gang. Many are on the tower; some are on the scaffolding, and one is at the base of the tower. There is some religious equipment on the top of the tower too, different from what the heroes saw at the consecration ceremony. There are no guards or watchmen patrolling the fence. When the heroes arrive, Akrash is persuading Manakenus that his path to salvation and inner peace is through Illumination, and that this can be achieved by severing his final tie to mundane things: he must throw his baby daughter from the tower. Unless the heroes stop them, this is what happens. As the heroes rush up the scaffolding to Manakenus, the Broken Knee gangers rush down and a fight ensues on the scaffolding. There are about as many gang members as heroes, but the gang starts with 17AP for each hero; AP losses can be represented by gangers falling from the scaffolding. If the fight goes badly for the gang, Akrash will use his Lunar magic to lend them APs. If it's needed, the protesters appear at the bridge during the fight. How they were treated earlier by the heroes will determine how much assistance they lend them now, but the protesters are eager to stop Akrash and Manakenus, as they think that the ceremony has something to do with completing the bridge. Once the gang has been defeated, the heroes enter a contest with Akrash over the fate of Manakenus's baby. He wants Manakenus to throw down the baby; the heroes, presumably, don't. If the heroes succeed, Manakenus sees the error of his ways and breaks down in tears while Akrash flees. If the heroes fail, Manakenus throws his child to her death (unless the heroes can somehow save her) and then collapses in a state of religious rapture. === Scene 2 (Epilogue): Cut Scene: Akrash's reward === ''Key points: Akrash receives his just rewards. '' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! After the players have congratulated or consoled each other, the scene cuts to Akrash a few days later (the heroes are not present). ! |} He is in a plain office in the Mayor's palace, being interviewed by a Lunar minister about the events surrounding Manakenus and the bridge. If the heroes failed and Akrash persuaded Manakenus to sacrifice his child, the minister praises Akrash for a job well done and comments that the bridge now has a great champion in Manakenus, the flawed Illuminate; he will commit himself, body and soul, to completing the bridge. Akrash is presented with the Order of the Glory of the Embracing Moon, First Class. Five years later, the bridge is completed and Raibanth becomes united once more. If, on the other hand, Akrash was foiled, the minister he is seen berating him and blaming him for the probable failure of the bridge. The minister wonders if Akrash's talents would better serve the Empire with thirty years of hard labour in a Danfive Xaroni gulag in the Redlands. Akrash is never seen again. In Raibanth, the work on this bridge, like so many others, peters out after less than three years of turmoil and strife and it remains forever barely started. == Locations == === Raibanth === Glorious Raibanth is the ancient Solar capital of Dara Happa, predating the rise of the Red Moon, the incorporation of the Dara Happans in the Lunar Empire, and the creation of a new capital at Glamour. It is a proud and ancient city, with a population of some 75,000 souls, dominated by the huge gilded statue of Raiba, its mythical founder, and the ziggurat of the Overseer. The city is circular, following the orderly patterns laid down by those who follow Yelm, the Imperial Sun, but it is divided into three parts by the Oslir and Joat Rivers. New and Old Raibanth, on the western and eastern banks of the Oslir respectively, are dominated by the Yelmics, although there is a growing body of Lunar worshippers in New Raibanth in particular. They are joined by just the Bridge of Seven Saints, and although this is a huge span, across which six ox-carts can rumble side-by-side, this nonetheless limits the contact between New and Old Raibanth. Ever since the destruction of the Bridge of Broken Angels’ Wings shortly before the Birth of the Goddess, no bridges connect the main city to Amoli, on the other side of the Joat. This is likely to remain the case for some time, as Amoli is largely populated by Darjiini, resentful of their Dara Happan masters and determined to retain as much independence as they can. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Dara Happa is not Rome, nor is it Babylonia, nor is it Byzantium, although it displays some of the traits of all three. Especially for the purpose of a demonstration game, where players may well know nothing about Glorantha, it may be easiest to treat the city as being quite Roman in feel, although it is worth noting that this is an over-simplification. ! |} === The Bridge === This is the latest of several projects to span the Oslir river and reunite New and Old Raibanth; the latter is a hard-line Solar community which resists integration into the Lunar Empire. To the Lunar government, the bridge is a symbol of healing and inclusion in the Empire. The bridge will span the mighty Oslir in several arches. At the moment, construction on the bridge has barely started. The first of many support columns has been built 15 metres high on the river's edge. The first few stones of the roadway are placed atop this column in the ceremony of Act 1 Scene 2. The column remains covered in scaffolding and the area surrounding it is a building site, littered with stone blocks in various states of completion, piles of sand and gravel, various tools (some even locked in the numerous toolsheds). The site is surrounded by a fence and normally patrolled by private watchmen. On the scaffolding itself, there is all the paraphernalia of building work, just waiting to be used by or against the Broken Knee gang members. Hammers, chisels, ropes, buckets (some full of various liquids and powders), piles of stones and mortar, and the like all lie around the many levels of the scaffolding. Some of the flooring planks and the ladders are loose, ready to trip the unwary, and many more can be loosened quickly. Ropes slung over pulleys are everywhere, some attached to heavy loads (around the weight of a person), and all ready to be freed in an instant. In other words, there should be plenty of props to help you come up with an entertaining fight on the bridge column. === Manakenus's villa === Manakenus lives in a middling-sized villa in a well-to-do part of New Raibanth, as befits a man of his position. A high wall surrounds a spacious formal garden. The main double gate is on the main road; a smaller, single door opens out onto an alley at the rear. The villa is a circular, single story ring of rooms surrounding a central courtyard. In the courtyard is a small obelisk, a shrine to Yelm. The villa has numerous rooms. The reception rooms are at the front of the house, and large doors that open onto the courtyard. Private rooms are on the sides. Servants’ quarters, kitchens, and storerooms are at the back. Inside, the floors are tiled and the walls are wood panelled and covered in hanging rugs. Various objets d'art adorn the villa, hanging on the walls, in cabinets, or on plinths. == Dramatis Personae == === Manakenus === ''Age 34, male, architect, initiate of Yelm the Overseer and Hwarin Dalthippa the Builder. '' Manakenus is a prominent Tenth in the Shining Ray Column League (the same league as the heroes). He has recently been made Overseer of the construction of the new bridge, a high honour for one so young. However, the appointment has weighed heavy on his mind and he has begun to crack under the strain. He has come to rely on Akrash, and his eccentric brand of philosophy, for support through these times. He has a wife (Delra) and four-month-old daughter (Amena); Chaldra, a nanny, looks after Amena. Main Abilities: Civil engineer 17WWW, High class 2W, Oversee workers 3WW, Overseer magic 12W, Sedenyic philosophy 8W. === Akrash === ''Age 68, male, heretical Lunar holy man, devotee of Rashorana and many other aspects of the Red Moon, Illuminated (according to him). '' Akrash is an independent seeker after the Truth that lies behind the Masks of the Red Moon. He always wears a half-mask himself. His theories on Rashorana, Illumination, and other matters are out of line with mainstream Lunar practices, but he continues to practice and preach his version of the religion from his small church. He has recently become a great friend and confident of Manakenus and is with him a great deal. He is convinced that Manakenus's salvation lies in his own brand of ‘Illumination’ – entailing sacrificing all that matters to him in the mundane world that he can concentrate exclusively on Higher Truths. He is also an operative of the Spoken Word, one of the Instrumentalities (Lunar secret police agencies), or rather a dupe – he is under such deep cover that even he's convinced that he's a free agent. Main Abilities: Pious 4WW, Devoted to Rashorana 12WW, Own Brand of Sedenyic Philosophy 3WW, Persuasive 17W, Forceful personality 10W. Healing magics 2WW, Madness magics 18. === The Broken Knee gang === ''Small-time roughs '' The Broken Knee is a small gang of street heavies. Their territory includes Akrash's church, and he is paying them protection money, and a bit extra to act as his muscle. Typical gang member: Cudgel combat 17 (+3), Avoid Clever Stratagem 10, Resist Magic 14, Athletic 16, Relationship to Akrash 15. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Gang members should use their Avoid Clever Stratagem ability as their resistance to being tricked our outwitted by the heroes. ! |} === The Solar Protesters === ''Old Yelmic protesters. '' This is a small group of protesters from Old Raibanth. They have come over the Oslir to protest against, and hopefully stop, the construction of the new bridge. They are not violent, but will defend themselves. They are led by Murhardavu, an ageing Yelmic patriarch. They should be slightly comical, although possibly useful in the closing fight. Typical protester: Cudgel Combat 15 (small club +2), Chant Long-Winded Protest 2W, Conservative 18, Naïve 16. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] fd7b2fccc1e3b0a0d81613b9b189cbbd101882c5 1926 1923 2009-01-30T22:38:22Z Neil 2 Changed download link wikitext text/x-wiki {|align="right" style="width:50%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |You can download a formatted version of the [[:Image:Light-and-Death-episode.pdf|scenario (PDF, 895 Kb)]]. |} <blockquote>A [[:Category:HeroQuest|HeroQuest]] scenario by Neil Smith</blockquote> <blockquote>Edited by Mark Galeotti</blockquote> <blockquote>This scenario is copyright © Neil Smith 2003, 2004.</blockquote> Note that this scenario was written before the ''Imperial Lunar Handbook: Volume II'' was written. Some parts of the scenario, particularly details of Illumination, have been superceded by that book. But things are still close enough for a bit of a romp. == Rules and Benchmarks == This episode is designed to be played in a single session of perhaps 3-4 hours. Six pre-generated characters are provided [[Light and Death characters|in a separate page]]. The episode is based in the city of New Raibanth in the Lunar Heartlands. It shouldn't be too hard to transplant it to other Lunar cities, but some work will be needed. You will need a copy of the HeroQuest core rulebook. It would also be very helpful to have access to Imperial Lunar Handbook 1: The Lunar Empire, {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Sections shaded and enclosed with exclamation marks are notes for using the episode as a demonstration. ! |} This episode stresses investigation and clever solutions. Heroes' best abilities should be around 1W to 8W. Contests are described in a short form, as '''task (resistance)''': ''typical appropriate ability/ies (with suggested improvisational penalties)'', for example '''realise that the city magistrates could become involved (14)''': ''Dara Happan Customs''. Clearly, many other abilities may be used, and the narrator ought to rule where necessary as to what abilities are appropriate and what kind of improvisational penalties (if any) need to be applied. == Overview == Manakenus the Architect is a prominent member of the Shining Column Ray League of Raibanth. He has recently been charged with overseeing the construction of a new bridge across the Oslir river to further unite the cities of Old and New Raibanth. However, the stress of this job has forced Manakenus to listen to the teachings of Akrash, a Lunar fringe cultist. Akrash is offering the prospect of releasing Manakenus from the stress he is under by leading him to a form of Illumination. Once Illuminated, Manakenus will be free from all the bonds that cause his mental grief. Akrash starts this process by killing Manakenus’s wife. The heroes investigate and initially follow the clues that point to a group of Old Raibanth protesters. However, when they find the protesters, the heroes discover new evidence that points to Akrash. When they try to find him, they encounter the Broken Knee gang who are eventually persuaded to reveal that Akrash and Manakenus, together with his baby daughter, have gone to the bridge. The heroes rush to the scene and hopefully prevent Manakenus killing his daughter to achieve Illumination. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Such weighty theological issues as the worship of the transcendental Lunar goddess Rashorana and Illumination will be explored in future Imperial Lunar Handbooks, and will not take the form described here. The ‘Illumination’ Akrash peddles is more like a charismatic’s con trick, and not genuine Lunar Illumination. Likewise, the magic powers he manifests are not the real ones which will be available to those who follow Rashorana (whom, some say, grants no magic at all!), but reflect some strange cult of which he has been a member. ! |} == Act 1: Introduction == === Scene 1: To the Bridge! === ''Key points: The heroes help Manakenus escape from his villa, through a ring of fanatics. '' The heroes are at the villa of Manakenus, the chief architect of the New Oslir Bridge, on the morning of the official consecration of the building work. The heroes have been charged, by their League's patriarch, to ensure that Manakenus gets to the ceremony safe and sound. Just as they are preparing to leave, a group of protesters appears, obviously trying to prevent Manakenus from attending the ceremony. The heroes have to get themselves, Manakenus, and an associate of his by the name of Akrash, out of the villa. Manakenus is scared of the protesters, and doesn't want to leave the villa. The heroes may be able to '''overcome his fears (17)''': ''Persuasive, Command civilian (-5)'', but if not, then Akrash manages to persuade him that his duty lies at the ceremony. "These yelmies," he says, "may worship the Light, but they do not See. Your bridge, by spreading the gentle glow of Rufelza into their city and their hearts, will Heal them of their closed-minded refusal to accept what is new and better. You cannot allow them to defeat you, by keeping you away from your rightful place." A few heavies from the Shining Column Ray League guard the villa, and will prevent any of the protesters breaking into the villa. They will not stray from the villa or its grounds. It is up to the heroes to get Manakenus to the bridge. They have some choices about how to do it. Fighting the protesters is possible, and the heroes will have the assistance of villa’s guards. However, the heroes may '''realise that the city magistrates could become involved (14)''': ''Dara Happan Customs'', and their levy-men would likely arrest whoever started the fight. The protesters could be persuaded to leave, or let Manakenus leave, but the protesters are determined to stay (resistance of 20). Finally, the heroes could sneak out the back way, if they can persuade Manakenus that a man of his station should stoop to such measures. The protesters follow Manakenus and the heroes when they leave. === Scene 2: The Consecration === ''Key points: The heroes watch the consecration ceremony. '' The heroes arrive at the site of the bridge, for the ceremony that celebrates the laying of the first stone of the roadway. Manakenus climbs the first column, where he joins various bigwigs from the Raibanth and Lunar governments. Akrash goes with him, but the heroes soon lose sight of him. Sacrifices are made to Yelm, Sedenya, and Hwarin Dalthippa (a Lunar goddess associated with engineering and construction) and they are asked to bless and protect the bridge. A crowd watches the consecration, with some heckling from scattered groups of protesters. There is little, if anything, for the heroes to do here, so this scene is best skipped over quickly. However, if you want to extend this section, then a few options would be for a protester to try to make trouble or for the heroes to spot a pick-pocketing in the crowd (do they leave Manakenus’s side and intervene?). === Scene 3 (Plot Turn 1): A Murder is Discovered === ''Key points: On returning to his villa, the heroes discover Manakenus's wife murdered. They vow, and are asked, to investigate. '' After the consecration, Manakenus sends a message to heroes that he will be staying at the bridge for some time, and that the heroes should return to his villa for the pre-arranged formal meal. When they return to the villa, they find the front gate ajar and unguarded. The watchman lies dead, half-concealed under a bush. Inside the house, the heroes can hear sobbing. There are bloodstained footprints in the hall, made by several feet, and they lead out, down the path to the gate. In the main reception room, Delra (Manakenus's wife) lies messily murdered, with blood and brains spattered across the walls. A blood-soaked cudgel, the murder weapon, lies nearby. In the kitchen, the cook has also been killed, stabbed and bludgeoned. The sobbing comes from a bedroom where Chalda, the family's nanny, has barricaded herself with Amena, Manakenus's child. She must be '''persuaded to come out (18)''': ''persuade, issue order (-8), intimidate (-15)'', but both she and the baby are unharmed. The villa is otherwise deserted. The heroes will need to '''calm Chalda down (14)''': ''Comfort, Persuade, Bargain (-5)''; if successful, she will gladly tell the heroes her story, through her sobs. After Manakenus and the heroes left, the protesters also left, shadowed by the guards. Chalda decided to take Amena out for a walk, and they were gone for about an hour. When they returned, she glimpsed someone in yellow, Yelmic robes darting out of the villa gate. She came into the villa, saw Delra dead, and then hid in the bedroom until the heroes arrived. She is of no further use to the heroes, and will gratefully be sent off to another house in the League. The heroes should now realise that it is their duty to find the perpetrators of this crime and bring them to justice. If they don't, other members of the League make it plain that doing so is their duty. == Act 2: Confrontation == === Scene 1: Finding the Suspects === ''Key points: The heroes track the protesters to their hideout. '' As the protesters are the obvious suspects, the heroes need to find them first. Asking passers-by in the street will lead nowhere: no one saw the murderers escape. The owner of a fried sausage stall will remember a group of solar protesters returning to the villa, but they left hurriedly almost immediately after. He didn't know where they were going, but did overhear someone shouting about returning to the docks. '''Asking around after the protesters (14)''': ''Know Raibanth, Relationship to League (-5), (searching the docks +5)'' will lead the heroes to a boarding house near the docks on the Oslir. The owner, a fat, drunken slob, will, if '''persuaded (10)''': ''persuade, fast talk, wealth'' confirm that the top floor of the five-story block has been booked by "a bunch of whinging, snobby types from over th' rivuh. They're upstairs now" === Scene 2: Struggle in the Sunlight === ''Key points: The heroes get to question the protesters. '' Assuming the heroes don't take too long, the protesters will not be in their rooms. Instead, the heroes will find them gathered in discussion on the flat roof of the boarding house. They are terrified that they will be accused of the murders at Manakenus's villa, and are on the verge of cutting and running back to Old Raibanth. They're nervous and jumpy, but they won't initiate hostilities. A couple of protesters are guarding the stairs to the roof and will try to prevent the heroes, or anyone else, getting onto the roof. The appearance of the heroes will cause consternation among them, increased when they realise that the heroes are from Manakenus's League. The heroes can persuade them to talk, beat them into submission, or employ some combination of the two (use an extended contest; see the protesters’ statistics at the end). The roof is mainly flat, with some trapdoors for access, a shed for tool storage, and various lumps, bumps, and low walls. If the protestors are defeated, they are prepared to tell their story. If things are going too well for the heroes, maybe the protesters are not as feeble as they appear, or perhaps their leader, Murhardavu, has some unexpected magical power? Maybe he can Manifest Yelmic Authority 155, forcing Dara Happan characters to kneel before him? Or perhaps the boarding house has other tenants who don’t take kindly to heavy-handed heroes storming in? === Scene 3 (midpoint): Whose Pendant is This? === ''Key points: The heroes learn that the protesters did not murder Manakenus's wife. They did pick up a talisman dropped by the murderer, though, and the heroes recognise it as belonging to Akrash. '' Once the situation on the rooftop has calmed, the protesters will tell their story. They followed the heroes to the consecration ceremony, but were moved on by guards there. They then decided to return to Manakenus's villa, intending to set up a protest for when he returned. However, when they arrived at the villa, they found the gate open and decided to go in. They found Manakenus's wife already dead, and fled. One of their number picked up a blood-spattered silver pendant marked with Lunar symbols lying near the wife's body. On examining the pendant, they should recognise it as Akrash's (16): Keen eye, Sedenyic philosophy. On listening to the protesters’ story, the heroes should '''realise that their story is true (14)''': ''Detect lies, Discern truth, Persuade (-5)''. If the heroes were defeated by the solars, they are forced to retreat, but notice (and perhaps purloin) Akrash's pendant lying on the floor. === Scene 3a: Meanwhile... === ''Key points: What Akrash and Manakenus are doing. '' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Note: this “scene” is for Narrator information only, not for the heroes to play through. You may want to use it as a “cut scene” if you and the players would enjoy it. ! |} While the heroes are tracking down the protesters, Akrash and Manakenus are not idle. Manakenus stays at the consecration ceremony for a while, until he hears of the murder of his wife. He then rushes home to see her, and is taken to where his baby and nanny are staying. He stays with them for a few hours, until Akrash turns up to offer him comfort and support. While they are alone, Akrash persuades Manakenus that he will only be free of his pain by severing his remaining ties and becoming Illuminated. Manakenus agrees, and takes his baby to Akrash's church where they pray for a while. After nightfall they, together with some Broken Knee heavies, go to the bridge site for the final confrontation. === Scene 4 (Plot Turn 2): At the Church === ''Key points: The heroes rush to Akrash's church, where they are confronted by a number of the Broken Knee gang. They learn that Akrash, Manakenus, and his baby daughter have left for a special ceremony on the new bridge.'' If the protesters are not to blame for the murders, who is? The trail points to Akrash, so the heroes will have to locate him. If they check, he is not at the villa or with Manakenus's nanny, though Chalda will say that Akrash and a distraught Manakenus left with the baby "to liberate themselves from grief" a few hours ago. She thinks they were going to Akrash's church, but they did also mention the bridge as the cause of all Manakenus's grief. The heroes may '''know the location of Akrash's church (18)''': ''Know Raibanth, Sedenyic Philosophy (-5), Initiate of Whoever (-12)''. Some of Manakenus's servants know where it is, or they can '''ask around in the city (18)''': ''Dara Happan customs, Underworld contacts (-5)''. The church is a converted shop in a residential area of the city near the docks. Outside, it is painted with Lunar symbols. When the heroes arrive, Akrash and Manakenus have already left, but three Broken Knee gang members are hanging around outside, guarding it. They'll try to scare off the heroes, and won't be too averse to using force if it's needed. If they look like they're losing the encounter, they will retreat into the church and lock the doors. The heroes may need to either break or sneak in finally to defeat the guards. When the guards are finally defeated and '''questioned (18)''': ''Discern truth, Persuade (-3)'', they will grudgingly confirm that Akrash, "some snotty Tenth [minor aristocrat] and a babe-in-arms" were at the church and prayed for a while, and then left for the site of the new bridge to perform some ceremony. The rest of the gang went with them. == Act 3: Resolution == === Scene 1: Going Up and Down === ''Key points: The heroes must fight their way up the scaffolding, through the rest of the Broken Knee gang, to prevent Akrash's nefarious plot. They are either helped or hindered by the remains of the protesters. '' It is now (or should be) night. The heroes rush to the site of the new bridge, and see a number of people atop the first tower, including Akrash and Manakenus. Manakenus is holding his baby. The rest of the people are members of the Broken Knee gang. Many are on the tower; some are on the scaffolding, and one is at the base of the tower. There is some religious equipment on the top of the tower too, different from what the heroes saw at the consecration ceremony. There are no guards or watchmen patrolling the fence. When the heroes arrive, Akrash is persuading Manakenus that his path to salvation and inner peace is through Illumination, and that this can be achieved by severing his final tie to mundane things: he must throw his baby daughter from the tower. Unless the heroes stop them, this is what happens. As the heroes rush up the scaffolding to Manakenus, the Broken Knee gangers rush down and a fight ensues on the scaffolding. There are about as many gang members as heroes, but the gang starts with 17AP for each hero; AP losses can be represented by gangers falling from the scaffolding. If the fight goes badly for the gang, Akrash will use his Lunar magic to lend them APs. If it's needed, the protesters appear at the bridge during the fight. How they were treated earlier by the heroes will determine how much assistance they lend them now, but the protesters are eager to stop Akrash and Manakenus, as they think that the ceremony has something to do with completing the bridge. Once the gang has been defeated, the heroes enter a contest with Akrash over the fate of Manakenus's baby. He wants Manakenus to throw down the baby; the heroes, presumably, don't. If the heroes succeed, Manakenus sees the error of his ways and breaks down in tears while Akrash flees. If the heroes fail, Manakenus throws his child to her death (unless the heroes can somehow save her) and then collapses in a state of religious rapture. === Scene 2 (Epilogue): Cut Scene: Akrash's reward === ''Key points: Akrash receives his just rewards. '' {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! After the players have congratulated or consoled each other, the scene cuts to Akrash a few days later (the heroes are not present). ! |} He is in a plain office in the Mayor's palace, being interviewed by a Lunar minister about the events surrounding Manakenus and the bridge. If the heroes failed and Akrash persuaded Manakenus to sacrifice his child, the minister praises Akrash for a job well done and comments that the bridge now has a great champion in Manakenus, the flawed Illuminate; he will commit himself, body and soul, to completing the bridge. Akrash is presented with the Order of the Glory of the Embracing Moon, First Class. Five years later, the bridge is completed and Raibanth becomes united once more. If, on the other hand, Akrash was foiled, the minister he is seen berating him and blaming him for the probable failure of the bridge. The minister wonders if Akrash's talents would better serve the Empire with thirty years of hard labour in a Danfive Xaroni gulag in the Redlands. Akrash is never seen again. In Raibanth, the work on this bridge, like so many others, peters out after less than three years of turmoil and strife and it remains forever barely started. == Locations == === Raibanth === Glorious Raibanth is the ancient Solar capital of Dara Happa, predating the rise of the Red Moon, the incorporation of the Dara Happans in the Lunar Empire, and the creation of a new capital at Glamour. It is a proud and ancient city, with a population of some 75,000 souls, dominated by the huge gilded statue of Raiba, its mythical founder, and the ziggurat of the Overseer. The city is circular, following the orderly patterns laid down by those who follow Yelm, the Imperial Sun, but it is divided into three parts by the Oslir and Joat Rivers. New and Old Raibanth, on the western and eastern banks of the Oslir respectively, are dominated by the Yelmics, although there is a growing body of Lunar worshippers in New Raibanth in particular. They are joined by just the Bridge of Seven Saints, and although this is a huge span, across which six ox-carts can rumble side-by-side, this nonetheless limits the contact between New and Old Raibanth. Ever since the destruction of the Bridge of Broken Angels’ Wings shortly before the Birth of the Goddess, no bridges connect the main city to Amoli, on the other side of the Joat. This is likely to remain the case for some time, as Amoli is largely populated by Darjiini, resentful of their Dara Happan masters and determined to retain as much independence as they can. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Dara Happa is not Rome, nor is it Babylonia, nor is it Byzantium, although it displays some of the traits of all three. Especially for the purpose of a demonstration game, where players may well know nothing about Glorantha, it may be easiest to treat the city as being quite Roman in feel, although it is worth noting that this is an over-simplification. ! |} === The Bridge === This is the latest of several projects to span the Oslir river and reunite New and Old Raibanth; the latter is a hard-line Solar community which resists integration into the Lunar Empire. To the Lunar government, the bridge is a symbol of healing and inclusion in the Empire. The bridge will span the mighty Oslir in several arches. At the moment, construction on the bridge has barely started. The first of many support columns has been built 15 metres high on the river's edge. The first few stones of the roadway are placed atop this column in the ceremony of Act 1 Scene 2. The column remains covered in scaffolding and the area surrounding it is a building site, littered with stone blocks in various states of completion, piles of sand and gravel, various tools (some even locked in the numerous toolsheds). The site is surrounded by a fence and normally patrolled by private watchmen. On the scaffolding itself, there is all the paraphernalia of building work, just waiting to be used by or against the Broken Knee gang members. Hammers, chisels, ropes, buckets (some full of various liquids and powders), piles of stones and mortar, and the like all lie around the many levels of the scaffolding. Some of the flooring planks and the ladders are loose, ready to trip the unwary, and many more can be loosened quickly. Ropes slung over pulleys are everywhere, some attached to heavy loads (around the weight of a person), and all ready to be freed in an instant. In other words, there should be plenty of props to help you come up with an entertaining fight on the bridge column. === Manakenus's villa === Manakenus lives in a middling-sized villa in a well-to-do part of New Raibanth, as befits a man of his position. A high wall surrounds a spacious formal garden. The main double gate is on the main road; a smaller, single door opens out onto an alley at the rear. The villa is a circular, single story ring of rooms surrounding a central courtyard. In the courtyard is a small obelisk, a shrine to Yelm. The villa has numerous rooms. The reception rooms are at the front of the house, and large doors that open onto the courtyard. Private rooms are on the sides. Servants’ quarters, kitchens, and storerooms are at the back. Inside, the floors are tiled and the walls are wood panelled and covered in hanging rugs. Various objets d'art adorn the villa, hanging on the walls, in cabinets, or on plinths. == Dramatis Personae == === Manakenus === ''Age 34, male, architect, initiate of Yelm the Overseer and Hwarin Dalthippa the Builder. '' Manakenus is a prominent Tenth in the Shining Ray Column League (the same league as the heroes). He has recently been made Overseer of the construction of the new bridge, a high honour for one so young. However, the appointment has weighed heavy on his mind and he has begun to crack under the strain. He has come to rely on Akrash, and his eccentric brand of philosophy, for support through these times. He has a wife (Delra) and four-month-old daughter (Amena); Chaldra, a nanny, looks after Amena. Main Abilities: Civil engineer 17WWW, High class 2W, Oversee workers 3WW, Overseer magic 12W, Sedenyic philosophy 8W. === Akrash === ''Age 68, male, heretical Lunar holy man, devotee of Rashorana and many other aspects of the Red Moon, Illuminated (according to him). '' Akrash is an independent seeker after the Truth that lies behind the Masks of the Red Moon. He always wears a half-mask himself. His theories on Rashorana, Illumination, and other matters are out of line with mainstream Lunar practices, but he continues to practice and preach his version of the religion from his small church. He has recently become a great friend and confident of Manakenus and is with him a great deal. He is convinced that Manakenus's salvation lies in his own brand of ‘Illumination’ – entailing sacrificing all that matters to him in the mundane world that he can concentrate exclusively on Higher Truths. He is also an operative of the Spoken Word, one of the Instrumentalities (Lunar secret police agencies), or rather a dupe – he is under such deep cover that even he's convinced that he's a free agent. Main Abilities: Pious 4WW, Devoted to Rashorana 12WW, Own Brand of Sedenyic Philosophy 3WW, Persuasive 17W, Forceful personality 10W. Healing magics 2WW, Madness magics 18. === The Broken Knee gang === ''Small-time roughs '' The Broken Knee is a small gang of street heavies. Their territory includes Akrash's church, and he is paying them protection money, and a bit extra to act as his muscle. Typical gang member: Cudgel combat 17 (+3), Avoid Clever Stratagem 10, Resist Magic 14, Athletic 16, Relationship to Akrash 15. {|style="width:100%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |! Gang members should use their Avoid Clever Stratagem ability as their resistance to being tricked our outwitted by the heroes. ! |} === The Solar Protesters === ''Old Yelmic protesters. '' This is a small group of protesters from Old Raibanth. They have come over the Oslir to protest against, and hopefully stop, the construction of the new bridge. They are not violent, but will defend themselves. They are led by Murhardavu, an ageing Yelmic patriarch. They should be slightly comical, although possibly useful in the closing fight. Typical protester: Cudgel Combat 15 (small club +2), Chant Long-Winded Protest 2W, Conservative 18, Naïve 16. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] efba8b233fff5c76031e07bf1d9b78de24554be7 0 1566 1924 2009-01-30T22:33:14Z Neil 2 New page: {|align="right" style="width:50%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Download formatted versions of the [[:Image:Light-and-Deat... wikitext text/x-wiki {|align="right" style="width:50%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |Download formatted versions of the [[:Image:Light-and-Death-episode.pdf|scenario (PDF)]] and [[:Image:Light-and-Death-characters-ww.pdf|characters (PDF)]]. |} These are the pre-generated characters for the [[Light and Death]] episode for [[:Category:HeroQuest|HeroQuest]]. == Labgatha: Yelmic patriarch== You are a senior Tenth in your League, and take your position very seriously. You are descended from Yelm, the first Emperor, and you do your best to Rule with the same clarity and sureness that he did. You take on the mantle of authority because it is your right and sacred duty to lead and command those under your care. It is their duty and honour to obey you in all things, for this is the Ancient and Eternal sacred Order of the world. {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Clear sight * Close Combat (Spear, club) * Dara Happan Customs * Dara Happan geography * Discern Truth 5w * Duelling * Endure * Exert authority 2w * Grooming * Identify social status * Issue order * Know Place in Society * Know Raibanth * Know Yelmic Law * Know Yelmic myths * Piercing glare * Poetry * Politics * Protocol * Raibanth Politics * Ranged combat (Bow, javelin) * Recite poetry * Refined Repartee * Ride * Urban Survival |width="50%"|'''Personality''' (all 17) * Conservative * Direct * Dutiful * Honourable * Strong Will '''Magic: Devotee of Antirius, Yelm the Emperor''' * ''Justice'' (Know Law, Question Witness, Enforce Judgement) 2w * ''Nobility'' (Regal Appearance, Command Commoner, Aura of Authority) 19 * ''Sky'' (Blinding Light, Welcome Glow, Heat Area, Clear Clouds) 17 '''Relationships''' (all 17) * To League * To other Patriarchs '''Equipment''' * Weapons * Writing equipment * Wealth 15w (coins, jewellery) * Expensive clothes '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Lerissa: Rufelzan convert/evangalist == First, there was the fierce light of Yelm. Then came Rufelza, who healed Yelm and now bathes the world in the gentle and inclusive light of the Red Moon. Rufelza taught that We Are All Us, and She is now spreading Her message to all the peoples of the world, so that they too may be healed. But her message has not been heard, even in the Heartlands of the Empire. You have dedicated your life to spreading the light and healing message of Rufelza to the Mobs of the Heartland Cities. Akrash is a great example of this missionary work, but you suspect that he may also be working for someone in Glamour (the Lunar capital). {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Attentive listener * Balance * Cajole * Confusing words * Cudgel combat * Debate * Embrace contradictions * Incite mob 19 * Know Dara Happan customs * Know Lunar customs * Know Lunar myths * Know Place in Society * Know Raibanth * Know Solar myths * Mental balance * Pass through mob * Persuade 5w * Powerful voice * Preach * Raibanth Politics * Sedenyic philosophy * Sense emotion * Urban Survival |width="50%"|'''Personality''' (all 17) * Inclusive 2w * Loyalty to Lunar Way * Zealous '''Magic''': Devotee of Rufelza, the Red Moon (all 17) * ''Healing'' (Cure Madness, Heal at Distance, Heal Self Beforehand, Heal Sickness) 2w * ''Understanding'' (Know Motivation, Predict Actions, Comprehend, Accept Contradiction) * ''Madness'' (Cure Madness, Make Dizzy, Make Scared, Mindblast) '''Relationships''' (all 17) * To League * To Lunar church * To Mob '''Equipment''' * Rufelzan religious paraphernalia * Flowing robes '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Vanyoradach: Lodrilli labourer (stevedore) == You are a labourer, working at the docks. You spend all day hauling bales, barrels, and sacks, and all night blowing off steam. You are like the volcano you worship: solid, dependable, fertile, but capable of huge explosions without warning. You really don’t like these stuck-up Yelmites, especially those whining Amoli from over the river, and especially when they tell you what to do. They all need a good slapping, and you’re the man to give it to them. Then you can go off and get pissed and get laid. {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Work all day * Party all night * Carry heavy load * Close combat (brawling, club, spear) 2w * Dara Happan Customs * Dara Happan geography * Dodge work * Fell off the back of a cart * Find a party * Haggle * Keep Warm * Know Place in Society * Know Raibanth * Know Solar Pantheon Myths * Look busy * Loom over opponent 5w * Raibanth Politics * Tell lewd jokes * Underworld contacts * Upthrusting Spear * Urban Survival |width="50%"|'''Personality (all 17) * Earthy * Explosive 19 * Hate Amoli * Proud * Resent being ordered about 18 * Resent Yelmites * Upfront * Needs things Explained '''Magic''': Initiate of Lodril, the Fire in the Earth (all 13) * Fire; Earth; Eruption * Common magic (16): Extra heave, Find the right crate, Barge through crowd, Fuming ears '''Relationships''' (all 17) * Family * League * Dockworkers 2w '''Equipment''' * Large club * Several crates that fell off the back of a barge '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Khorkenus: Second Story Man == Even respectable people need to get by when they fall on hard times. And really, We Are All Us, so the stuff you take is already yours, in a way. At least, that’s what you tell yourself sometimes. But really, you’re only in this for yourself, and it’s up to you to take what you can, what you need to survive. And it doesn’t really matter who you take it from. After all, no-one’s going to give it to you. And We Are All Us. {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Agile * Climb 2w * Cut deal * Evaluate victim * Find the back door * Guess value * Hide in shadows 5w * Hide loot * Knock out * Listen carefully * Look inconspicuous * Open lock * Predict watchman * Search room * See in the dark 2w * Word on the street |width="50%"|'''Personality''' (all 17) * Opportunistic * Sneaky 19 * Sly * Kleptomaniac 3w * Self-serving * Independent '''Common Magic (all 17)''' Avoid Bad Winds, Just Another Beggar, Dull Hunger, Hide in the Dark, Run In Crowds, Scamper over Wall, Convincing Liar '''Relationships''' (all 17) * League 8 * Thieves * Fences '''Equipment''' * B&E tools * Rope * Dark clothes '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Aneresh: Urvairinus Soldier (peltast) == You are a peltast, as skirmishing soldier in the Lunar Army. You have just returned from active duty in the Redlands, where you spent your time protecting settler villages from raiding horse nomads. Now you're back in home in Raibanth, you're looking forward to some rest, relaxation, and some fun outside the confines of military life. You're used to taking charge of civilians in {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Army Regulations * Climb * Close combat (brawling, scimitar, cudgel) 2w * Command civilians * Dara Happan Customs * Dara Happan geography * Identify foe * Know Place in Society * Know Raibanth * Know Solar Pantheon Myths * Look Busy * Make camp * March * Peltast mass combat * Peltast traditions * Raibanth Politics * Ranged combat (javelin, knife) 5w * Scouting 2w * Seen terrible things * Sneak * Surprise attack * Swear like a soldier * Urban Survival |width="50%"|'''Personality''' (all 17) * Haughty * Confident * Obey orders 18 '''Magic''': Initiate of Urvairinus the Soldier (all 17) * Peltast Combat * Destroy Dara Happan Foes * Light of Action 19 * Common magic: March further, Leap from cover, Hurl javelin, Tidy kit, Sharp eyes '''Relationships''' (all 17) * League * Soldiers '''Equipment''' * Javelins, scimitar, leather armour '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Jalvera: Etyries Merchant / fixer == You're a wheeler-dealer, a woman in a hurry, always moving on to the next deal. You know a lot of people, and you can get hold of almost anything, given enough time and silver. Your constantly search for a new deal, the next big thing, the big break that will make you rich. Your search takes you to some places that most people don't go, but your glib tongue allows you to get in, get what you want, and get out again. At least, most of the time. You’re convinced that there’s something going on there below the surface, and you want to find out what it is. There’s something not quite right about Akrash, though Lerissa seems to like him. {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Bargain * Carry load * Convince Buyer 5w * Cudgel combat * Dara Happan Customs * Dara Happan geography * Fast Talk * Find Trade * Know Place in Society * Know Raibanth * Know Solar Pantheon Myths * Lie 2w * Look unconcerned * Lunar Customs * Pretty * Raibanth Politics * Recognize value * Seek out bargain * Silver Tongue 19 * Solar Customs * Travelled Widely * Urban Survival |width="50%"|'''Personality''' (all 17) * Energetic * Greedy * Bubbly '''Magic''': Initiate of Etyries the trader (all 17) * Discovery 2w * Trade * Travel * Common magic: Brilliant smile, Look at this!, Discern buyer’s intent, Weigh coins, Find hidden thing '''Relationships''' (all 17) * To League * Underworld contacts * Traders * Street rumourmill '''Equipment''' * Trade goods * Horse + cart '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] 1288219d1ed91970add26227b96f71c7a0d0db59 1927 1924 2009-01-30T22:38:29Z Neil 2 Changed download link wikitext text/x-wiki {|align="right" style="width:50%; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" ||You can download a formatted version of the [[:Image:Light-and-Death-episode.pdf|scenario (PDF, 895 Kb)]]. |} These are the pre-generated characters for the [[Light and Death]] episode for [[:Category:HeroQuest|HeroQuest]]. == Labgatha: Yelmic patriarch== You are a senior Tenth in your League, and take your position very seriously. You are descended from Yelm, the first Emperor, and you do your best to Rule with the same clarity and sureness that he did. You take on the mantle of authority because it is your right and sacred duty to lead and command those under your care. It is their duty and honour to obey you in all things, for this is the Ancient and Eternal sacred Order of the world. {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Clear sight * Close Combat (Spear, club) * Dara Happan Customs * Dara Happan geography * Discern Truth 5w * Duelling * Endure * Exert authority 2w * Grooming * Identify social status * Issue order * Know Place in Society * Know Raibanth * Know Yelmic Law * Know Yelmic myths * Piercing glare * Poetry * Politics * Protocol * Raibanth Politics * Ranged combat (Bow, javelin) * Recite poetry * Refined Repartee * Ride * Urban Survival |width="50%"|'''Personality''' (all 17) * Conservative * Direct * Dutiful * Honourable * Strong Will '''Magic: Devotee of Antirius, Yelm the Emperor''' * ''Justice'' (Know Law, Question Witness, Enforce Judgement) 2w * ''Nobility'' (Regal Appearance, Command Commoner, Aura of Authority) 19 * ''Sky'' (Blinding Light, Welcome Glow, Heat Area, Clear Clouds) 17 '''Relationships''' (all 17) * To League * To other Patriarchs '''Equipment''' * Weapons * Writing equipment * Wealth 15w (coins, jewellery) * Expensive clothes '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Lerissa: Rufelzan convert/evangalist == First, there was the fierce light of Yelm. Then came Rufelza, who healed Yelm and now bathes the world in the gentle and inclusive light of the Red Moon. Rufelza taught that We Are All Us, and She is now spreading Her message to all the peoples of the world, so that they too may be healed. But her message has not been heard, even in the Heartlands of the Empire. You have dedicated your life to spreading the light and healing message of Rufelza to the Mobs of the Heartland Cities. Akrash is a great example of this missionary work, but you suspect that he may also be working for someone in Glamour (the Lunar capital). {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Attentive listener * Balance * Cajole * Confusing words * Cudgel combat * Debate * Embrace contradictions * Incite mob 19 * Know Dara Happan customs * Know Lunar customs * Know Lunar myths * Know Place in Society * Know Raibanth * Know Solar myths * Mental balance * Pass through mob * Persuade 5w * Powerful voice * Preach * Raibanth Politics * Sedenyic philosophy * Sense emotion * Urban Survival |width="50%"|'''Personality''' (all 17) * Inclusive 2w * Loyalty to Lunar Way * Zealous '''Magic''': Devotee of Rufelza, the Red Moon (all 17) * ''Healing'' (Cure Madness, Heal at Distance, Heal Self Beforehand, Heal Sickness) 2w * ''Understanding'' (Know Motivation, Predict Actions, Comprehend, Accept Contradiction) * ''Madness'' (Cure Madness, Make Dizzy, Make Scared, Mindblast) '''Relationships''' (all 17) * To League * To Lunar church * To Mob '''Equipment''' * Rufelzan religious paraphernalia * Flowing robes '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Vanyoradach: Lodrilli labourer (stevedore) == You are a labourer, working at the docks. You spend all day hauling bales, barrels, and sacks, and all night blowing off steam. You are like the volcano you worship: solid, dependable, fertile, but capable of huge explosions without warning. You really don’t like these stuck-up Yelmites, especially those whining Amoli from over the river, and especially when they tell you what to do. They all need a good slapping, and you’re the man to give it to them. Then you can go off and get pissed and get laid. {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Work all day * Party all night * Carry heavy load * Close combat (brawling, club, spear) 2w * Dara Happan Customs * Dara Happan geography * Dodge work * Fell off the back of a cart * Find a party * Haggle * Keep Warm * Know Place in Society * Know Raibanth * Know Solar Pantheon Myths * Look busy * Loom over opponent 5w * Raibanth Politics * Tell lewd jokes * Underworld contacts * Upthrusting Spear * Urban Survival |width="50%"|'''Personality (all 17) * Earthy * Explosive 19 * Hate Amoli * Proud * Resent being ordered about 18 * Resent Yelmites * Upfront * Needs things Explained '''Magic''': Initiate of Lodril, the Fire in the Earth (all 13) * Fire; Earth; Eruption * Common magic (16): Extra heave, Find the right crate, Barge through crowd, Fuming ears '''Relationships''' (all 17) * Family * League * Dockworkers 2w '''Equipment''' * Large club * Several crates that fell off the back of a barge '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Khorkenus: Second Story Man == Even respectable people need to get by when they fall on hard times. And really, We Are All Us, so the stuff you take is already yours, in a way. At least, that’s what you tell yourself sometimes. But really, you’re only in this for yourself, and it’s up to you to take what you can, what you need to survive. And it doesn’t really matter who you take it from. After all, no-one’s going to give it to you. And We Are All Us. {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Agile * Climb 2w * Cut deal * Evaluate victim * Find the back door * Guess value * Hide in shadows 5w * Hide loot * Knock out * Listen carefully * Look inconspicuous * Open lock * Predict watchman * Search room * See in the dark 2w * Word on the street |width="50%"|'''Personality''' (all 17) * Opportunistic * Sneaky 19 * Sly * Kleptomaniac 3w * Self-serving * Independent '''Common Magic (all 17)''' Avoid Bad Winds, Just Another Beggar, Dull Hunger, Hide in the Dark, Run In Crowds, Scamper over Wall, Convincing Liar '''Relationships''' (all 17) * League 8 * Thieves * Fences '''Equipment''' * B&E tools * Rope * Dark clothes '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Aneresh: Urvairinus Soldier (peltast) == You are a peltast, as skirmishing soldier in the Lunar Army. You have just returned from active duty in the Redlands, where you spent your time protecting settler villages from raiding horse nomads. Now you're back in home in Raibanth, you're looking forward to some rest, relaxation, and some fun outside the confines of military life. You're used to taking charge of civilians in {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Army Regulations * Climb * Close combat (brawling, scimitar, cudgel) 2w * Command civilians * Dara Happan Customs * Dara Happan geography * Identify foe * Know Place in Society * Know Raibanth * Know Solar Pantheon Myths * Look Busy * Make camp * March * Peltast mass combat * Peltast traditions * Raibanth Politics * Ranged combat (javelin, knife) 5w * Scouting 2w * Seen terrible things * Sneak * Surprise attack * Swear like a soldier * Urban Survival |width="50%"|'''Personality''' (all 17) * Haughty * Confident * Obey orders 18 '''Magic''': Initiate of Urvairinus the Soldier (all 17) * Peltast Combat * Destroy Dara Happan Foes * Light of Action 19 * Common magic: March further, Leap from cover, Hurl javelin, Tidy kit, Sharp eyes '''Relationships''' (all 17) * League * Soldiers '''Equipment''' * Javelins, scimitar, leather armour '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. == Jalvera: Etyries Merchant / fixer == You're a wheeler-dealer, a woman in a hurry, always moving on to the next deal. You know a lot of people, and you can get hold of almost anything, given enough time and silver. Your constantly search for a new deal, the next big thing, the big break that will make you rich. Your search takes you to some places that most people don't go, but your glib tongue allows you to get in, get what you want, and get out again. At least, most of the time. You’re convinced that there’s something going on there below the surface, and you want to find out what it is. There’s something not quite right about Akrash, though Lerissa seems to like him. {| |- valign="top" |width="50%"|'''Abilities''' (all 17) * Bargain * Carry load * Convince Buyer 5w * Cudgel combat * Dara Happan Customs * Dara Happan geography * Fast Talk * Find Trade * Know Place in Society * Know Raibanth * Know Solar Pantheon Myths * Lie 2w * Look unconcerned * Lunar Customs * Pretty * Raibanth Politics * Recognize value * Seek out bargain * Silver Tongue 19 * Solar Customs * Travelled Widely * Urban Survival |width="50%"|'''Personality''' (all 17) * Energetic * Greedy * Bubbly '''Magic''': Initiate of Etyries the trader (all 17) * Discovery 2w * Trade * Travel * Common magic: Brilliant smile, Look at this!, Discern buyer’s intent, Weigh coins, Find hidden thing '''Relationships''' (all 17) * To League * Underworld contacts * Traders * Street rumourmill '''Equipment''' * Trade goods * Horse + cart '''2 Hero Points''' |} === What You Know === You are all members of the Shining Column Ray League in the city of Raibanth in the Heartlands of the Lunar Empire. When the world was created, the Sun-Emperor, Yelm, ruled all the world from Raibanth in the rich lands of Dara Happa. His rule was firm (or harsh) but benevolent (for some). He was killed by the Final Rebel, but resurrected himself at the First Dawn. 400 years ago, a new Goddess was created. She was Rufelza, the Red Moon Goddess, and She spread a new message of peace, inclusion, and healing. Her followers defeated the Solar Empire and transformed it into the Lunar Empire. This Empire is now spreading across the world, bringing Rufelza's healing message to all. An important part of the Lunar religion is the idea that everything we perceive is but a Mask obscuring the Truth, but that the Truth is defined by the Mask we perceive. Lunars seek to find the Truth by seeking out many Masks, many points of view (preferably contradictory). Through understanding how these differ and are the same, the Lunar seeks the Illumination that the Masks, though vital, are unimportant and can be changed at will. Illumination leads to a person who becomes unfettered by any rules save what they choose to impose on themselves. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] 3fa3983825d368bf327bbe64d1afe8e5b2a9ef42 6 1567 1925 2009-01-30T22:36:27Z Neil 2 A formatted version of the [[Light and Death]] episode for [[:Category:HeroQuest]]. Includes pregenerated characters. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] wikitext text/x-wiki A formatted version of the [[Light and Death]] episode for [[:Category:HeroQuest]]. Includes pregenerated characters. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] 01891fc49d0f2ffdca85f43d2e3a997078a1a450 6 1561 1928 1893 2009-01-30T22:40:02Z Neil 2 Added category links wikitext text/x-wiki Formatted PDF of [[Peace Process characters|player characters]] for [[The Peace Process]] episode. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] dadb5706c27ebddb60fcd4029d10ebca7646764e 6 1560 1929 1892 2009-01-30T22:40:12Z Neil 2 Added category links wikitext text/x-wiki Formatted version of [[The Peace Process]] episode. [[Category:HeroQuest]] [[Category:Glorantha]] [[Category:Scenarios]] 1b05d31c574373d5612f6876f323f78d0d95c254 0 1568 1934 2009-02-03T21:56:18Z Neil 2 New page: == Context == This is an outline of a scenario I prepared for inclusion in a proposed HeroQuest scenario book, ''Sun Son City'', dealing with Lunar and Solar characters in the city of Raib... wikitext text/x-wiki == Context == This is an outline of a scenario I prepared for inclusion in a proposed HeroQuest scenario book, ''Sun Son City'', dealing with Lunar and Solar characters in the city of Raibanth. The book was to be a series of linked scenarios involving the trials and tribulations of a group of troubleshooters from the small and lower-class Association, the Broken Street Conjoinment. As things progressed, the PCs would be drawn further and further in the machinations between two large, rich, and powerful Associations, the Stardome Blessing and the Golden Line The overall arc was written by Mark Galeotti and Greg Stafford. However, with all the upheavals at Issaries in the past few years, the plans for the scenario book seem to have fallen by the wayside. This is the outline of the first scenario in the book. The intention was to get the PCs involved in various situations and places around the city of Raibanth, to gain some familiarity with the place. For that reason, it is unashamedly a wild-goose chase. The plot is structured as a three-layer onion. == Layer 1 == In the superficial layer, the PCs are approached (though their association) by a newly widowed young woman. Her husband, a night watchman at a spice dealer's warehouse, was killed a few days ago in a robbery. She wants the people that committed the offence to be brought to account, and to pay for what they did. She's also a bit doolally. The PCs are selected as they're go-getters in the Broken Street Conjoinment, and the widow has some connections with the association. She also has access to some funds to pay for the PCs help. Simple legwork will reveal that though the break-in may look like a robbery, things don't add up. Some of the stolen spices are found dumped nearby. Others are found swamping the local restaurant market in a different Association's territory, being sold for far below market price. There was cash on the premises, and much of this remains. The watchman seems to have been tortured before he was killed. There are some witnesses to the robbery, and they reveal that it wasn't a local gang that carried it out. Indeed, underworld contacts say that the raid was by the Strident Kidney Lion League, in a rather out-of-character (and out-of-turf) move. If they think about it, the widow's circumstances don't add up. Where did the funds come from to hire the PCs? Night watchman pay wouldn't be enough. There are two sources: the victim's other job, and funds being diverted to the widow by the Golden Line. The PCs were asked for by name, by the widow, for the job. This was because they were selected by the Golden Line. None of this is passed on to the PCs by their Association. And there are the cryptic comments made by the Pissing Man (see below). == Layer 2 == The PCs soon discover that robbery was not the real motive for the murder. Further investigation follows two avenues: why was the man killed, and why did the gang do it? The former question is answered when the PCs discover what the victim knew. What he had was the McGuffin (a meme; see below). He passed it to his wife before he died, and she has now passed it on. The Strident Kidney Lion are now hunting for the McGuffin, but don't really realize that they're looking for intangible information. While the PCs investigate, the Strident Kidney Lion continues to hunt down the information they need. They harass and beat up the widow, and vandalize their apartment. The victim's known associates are targeted, with actions ranging from offers of payments for information to vague threats and even beatings. The PCs will also have to protect the Strident Kidney Lion's various targets. Meanwhile, the McGuffin continues to pass from mind to mind, spreading mayhem and confusion in its wake. As the PCs learn more, there are hints of another layer of machinations. The McGuffin is high-power magic, and it seems unlikely that the Strident Kidney Lion are in a position to make use of it. Eventually, the PCs will discover that the Strident Kidney Lion were “hired” to find the McGuffin for an old, established Association, the Stardome Blessing. Of course, being and old, established Association, the Stardome Blessing didn't do anything as vulgar as pay the Strident Kidney Lion; instead, there is an understanding of favours for the Strident Kidney Lion in future. The PCs may also learn that their association knew all along about the connection to the Golden Line, but didn't tell them. They should start to question why, especially as it would have saved them from a potentially dangerous expedition for one particular clue. This section of the scenario will probably be presented as a set of clues for the PCs to discover, a set of priorities for the Strident Kidney Lion, and a proposed timetable for their actions. It is also likely to be the bulk of the scenario. == Layer 3 == Finally, the PCs have been able to identify the power behind the throne in the whole situation: the Stardome Blessing. The Stardome Blessing are too powerful for the Broken Street Conjoinment, let alone the PCs alone, to tackle, so the PCs may want to call in assistance. This allows them to have a confrontation with higher-ups in the Broken Street Conjoinment itself as they seek to discover who is pulling the strings in the Broken Street Conjoinment. After a short political struggle, the PCs will be introduced to someone poweful in the Golden Line (thought just how powerful is probably left unsaid). She will be willing to exchange information about the McGuffin for action against the Stardome Blessing. She will also explain, as clearly as a wacked-out, politicking, Lunar devotee can, that the obvious course of action may not be the correct one. It is up to the PCs (as a final test) to choose what retribution and recompense should be extracted. In doing this, the PCs get to define what the truth of the whole affair was. Perhaps it was a simple robbery-cum-murder after all... The current holder of the McGuffin will be taken away and made the subject of a powerful Lunar ritual, and all sorts of fun things could happen to them as a result! == Retribution and Recompense == The PCs must sort out these questions. Who bears responsibility for the murder (Strident Kidney Lion? Stardome Blessing?)? Should the recompense be in coin or corporal/capital punishment? Should it be proportionate, compensatory, or punitive? The choices the PCs make will have a significant effect on who starts out on top in the rivalry between the Golden Line and Stardome Blessing: choices here should be detailed in the "flowchart", though not explicitly explained to the players before the choices are made. == The Pissing Man == He is a recurring NPC. He is an agent for the Golden Line, and appears at odd moments to drop cryptic hints about events and avenues the PCs should explore (or avoid). The PCs should never learn his name, or really understand who he's working for. He gets his nickname because the only time the PCs see him is when they're in a public lavatory: the Pissing Man sits down beside the chosen PC, mutters something enigmatic while taking a leak, and disappears into the crowds before the PCs really have a chance to react. == The McGuffin == What this is is up for grabs, but I think it would be neat to have it take the form of a meme, transferable but not reproducible. It could take the form of a Nysalor riddle, which can only be answered by transferring it to another person (a la the videotape in “The Ring”). Contact with the meme also has profound effects on the holder's personality while they've got it, which linger even after it's been passed on. Perhaps this could link into Illumination, to show off that aspect of the world? However, whoever currently holds the meme also (subconsciously) holds the knowldge of how to open a portal to a Lunar short world (perhaps on the Red Moon?). Therefore, holding the person who holds the meme is important to both the Golden Line and Stardome Blessing. One idea is that the transmission of the meme is imperfect: as it moves from person to person, it becomes more powerful and dangerous. The first people that carry it suffer mild dementia while it's in them with few permanent after-effects. Later carriers suffer greater and greater efffects (schizophrenia, phychosis) and remain permanently mentally damaged after passing on the meme. The idea is to get the PCs very, very scared of the possibility of becoming a carrier of the meme, as well as showing how dangerous Illumination can be. That sets up a potentially neat final scene. The PCs finally track down the meme, which is residing in the now-shattered mind of a previously comfortable family man, last of a long line of people whose minds have been utterly destroyed by the meme. They have also identified a Lunar philosopher from the Golden Line. When the PCs bring the two together, she approaches the latest victim and listen to what he has to say, thus transferring the meme to her. She rises, turns to the PCs, says "How interesting," and walks off, seemingly completely unharmed. This could be used to reinforce that the deep truths of the Lunar Way are not for unprepared minds, and just how far removed from the common folk are the Illuminated leaders of Lunar thought. (Note: the McGuffin is vital to the success of the Golden Line's magical plans, which lead to the summoning of the entity in Scenario 4. In this way, the PCs' success here leads to the problems in scenario 3). [[Category:Scenarios]] [[Category:HeroQuest]] [[Category:Glorantha]] 9f905649a6e8941e16e67bb3e963817836e2bfc8 0 1464 1935 1593 2009-02-03T21:57:59Z Neil 2 wikitext text/x-wiki ''The Peaceful Cut ceremony is performed by Praxian nomads whenever they slaughter a herd-beast, and most other animals. It shows due deference to the spirit of the animal and guides it on its way back to Mother Erithra. This allows the spirit to be reborn into another animal, and also prevents the spirit haunting the butcher.'' The owner of the animal to be slaughtered (normally a woman) selects the animal from her herd. She approaches the animal and starts to sing a special song to it, to tell the animal what a good beast it is and all about the wonderful place it is going to. This song calms the animal and the woman leads it away to the waiting butcher. She will often give it a message for Mother Erithra just before she hands the animal over to the butcher. Meanwhile, the butcher has selected the spot where the animal will be killed. This should be out of site of the rest of the herds (a gully is ideal) and preferably is somewhere devoid of plants. He has assembled the tools he needs: the hobble-strap, the razor, the hoe, and the axe [1]. He takes the animal from the woman at the point where the animal can no longer see the herd. At this point, the animal is considered dead by the woman and the rest of the clan. The butcher takes the animal to the place of slaughter, singing a calming song. He hobbles the animal so that it cannot move and then chants to the animal to prepare it for its journey back to Mother Erithra [2]. When the time is right, the butcher says, 'The time has come for us to play our parts in the Covenant of Life and Death. We shall remember you in the wealth your body will give us. Return now to Erithra!' and slits the animal's throat with the razor [3]. The blood is allowed to spill to the ground, to provide the sustenance that will allow plants to grow in this spot in the future, to feed more animals. After a few seconds [4], the animal is pushed onto its side and its neck placed over a bowl to collect the rest of the blood. As the blood drains away, the animal's spirit becomes concentrated in its tail. Once the bleeding has stopped, the butcher uses the hoe to dig a pit in the blood-soaked earth and cuts off its tail with a single stroke of the axe. The tail is buried with a prayer to speed the animal's spirit to Mother Erithra. Once this is done, the butchering of the carcass can start. The animal is rolled onto its back and the first cut is along its belly from neck to tail. The skin is layed out to form a mat on which the butchering takes place; this is to prevent anything going to waste. [1] These tools of death are some of a Praxian man's most precious posessions. Good ones are passed down from father to son. [2] Rules note: this singing counts as Ceremony time to increase the butcher's Craft: Butchery skill, at a rate of one minute per increment. [3] Rules note: this is the casting of the Peaceful Cut spell. [4] The longer the animal is allowed to bleed onto the ground, the better luck the tribe will have, but the less food is obtained from the animal. In times of extreme distress, all the animal's blood is allowed to drain away. [[Category:Yellowtail]] [[Catgeory:Glorantha]] 15c51cd21b913c95ff298bf73e6d1ee348c86173 1936 1935 2009-02-03T21:58:24Z Neil 2 wikitext text/x-wiki ''The Peaceful Cut ceremony is performed by Praxian nomads whenever they slaughter a herd-beast, and most other animals. It shows due deference to the spirit of the animal and guides it on its way back to Mother Erithra. This allows the spirit to be reborn into another animal, and also prevents the spirit haunting the butcher.'' The owner of the animal to be slaughtered (normally a woman) selects the animal from her herd. She approaches the animal and starts to sing a special song to it, to tell the animal what a good beast it is and all about the wonderful place it is going to. This song calms the animal and the woman leads it away to the waiting butcher. She will often give it a message for Mother Erithra just before she hands the animal over to the butcher. Meanwhile, the butcher has selected the spot where the animal will be killed. This should be out of site of the rest of the herds (a gully is ideal) and preferably is somewhere devoid of plants. He has assembled the tools he needs: the hobble-strap, the razor, the hoe, and the axe [1]. He takes the animal from the woman at the point where the animal can no longer see the herd. At this point, the animal is considered dead by the woman and the rest of the clan. The butcher takes the animal to the place of slaughter, singing a calming song. He hobbles the animal so that it cannot move and then chants to the animal to prepare it for its journey back to Mother Erithra [2]. When the time is right, the butcher says, 'The time has come for us to play our parts in the Covenant of Life and Death. We shall remember you in the wealth your body will give us. Return now to Erithra!' and slits the animal's throat with the razor [3]. The blood is allowed to spill to the ground, to provide the sustenance that will allow plants to grow in this spot in the future, to feed more animals. After a few seconds [4], the animal is pushed onto its side and its neck placed over a bowl to collect the rest of the blood. As the blood drains away, the animal's spirit becomes concentrated in its tail. Once the bleeding has stopped, the butcher uses the hoe to dig a pit in the blood-soaked earth and cuts off its tail with a single stroke of the axe. The tail is buried with a prayer to speed the animal's spirit to Mother Erithra. Once this is done, the butchering of the carcass can start. The animal is rolled onto its back and the first cut is along its belly from neck to tail. The skin is layed out to form a mat on which the butchering takes place; this is to prevent anything going to waste. [1] These tools of death are some of a Praxian man's most precious posessions. Good ones are passed down from father to son. [2] Rules note: this singing counts as Ceremony time to increase the butcher's Craft: Butchery skill, at a rate of one minute per increment. [3] Rules note: this is the casting of the Peaceful Cut spell. [4] The longer the animal is allowed to bleed onto the ground, the better luck the tribe will have, but the less food is obtained from the animal. In times of extreme distress, all the animal's blood is allowed to drain away. [[Category:Yellowtail]] [[Category:Glorantha]] b6be83d8cbbcbc0e2877a1cee325bb1f335db6b3 0 1472 1937 1607 2009-02-03T21:58:41Z Neil 2 wikitext text/x-wiki (As told by Three Cows, Brave of the Yellowtail Clan) 'When the Lunar missionaries first came to Prax, the True People were unsure how to treat them. They were outsiders, dirtdiggers and so deserved contempt. But they had many gifts of metal to buy their way into tribes where they could use the magic of their clever tongues on the True People. But this could not hide the fact that their Red Queen did not follow the Covenant of Waha, and that she consorted with Chaos. These facts meant that most True People rejected the Red Men and killed them when they were found. In that way, we took their metal gifts without being corrupted by their clever tongues. But the Great Council of the Sable Nation heard too much of the Red Man's twisted words and agreed to help them, for many gifts of metal, magic, and salt. Only our Khan, Sun-Through-Clouds, spoke against them. He was forced out of the council tipi. 'In this way, when the Red Man's army came to Moonbroth, all the Sable Nation fought alongside them, except for the Yellowtail Clan, true followers of the Way of Waha. The battle went badly for the Red Men did not fight with honour. When the shamans summoned Oakfed, the Wildfire, the Lunars cast a terrible magic and sent Oakfed back against our shamans. Only because he was a stout follower of Waha and Daka Fal was Nine Sticks saved on that day, though he has not been right since. 'After the defeat of the True Peoples, the Sable Nation turned on our clan as traitors to the Nation. But they were the traitors! Bereft of allies, we were forced out of the good lands of Prax back out into the Wastes. Before we left, many men left and took their wives and herds with them, to join other Sable clans, may their names be cursed. 'That happened two-hands-and-one years ago. We have been in the Wastes all that time. Our clan is small. We are too small to prevent other tribes stealing our cows: that is why our herds are small. We are too small to aid Nine Sticks in driving away Malia: that is why our herds are sick, and why hoof-canker took many of them. That is why many babies died from the Shakes. We are too small to drive other clans from good grazing: that is why our cows are thin. Khan Sun-Through-Clouds has decided that we have spent too long in the Wastes, and that we must return to Prax, where we can steal cows, find good grazing, and bring up our children.' You can also read about the [[People of the Yellowtail Clan]]. [[Category:Yellowtail]] [[Category:Glorantha]] 2322073a7137ad03d0564f0f8eb2fb3dc56e6827 0 1473 1938 1609 2009-02-03T21:59:25Z Neil 2 wikitext text/x-wiki == Notable Personalities == === Khan Sun-Through-Clouds === (Wise, stoic, conservative) Khan Sun-Through-Clouds is almost the oldest man in the tribe, having lived for more than two-hands-of-hands of winters. He was once a great warrior and a wise Khan, well respected amongst the elders of the Sable Nation. But now he is old and not a great warrior. He once had a whole herd of captured beasts, a glorious thing, but he gave them to the tribe when times were hard and now he only owns two war sables, five impalas, two bison, four herd-men, and an ostrich. He also owns an iron-headed lance, a mighty bow, a stone that smells bad when Chaos is near, and an eagle head-dress. His lance is seldom used now and he can no more string the bow. It should belong to his sons, but they were killed at Moonbroth. His successor will take it when he dies. Sun-Through-Clouds has lead the tribe for many hands of years. He has lead the tribe well, protecting the herds and stealing cows from other tribes. He stood by his principles when the Sable Nation decided to join the Red Man's army, although it was an unpopular decision. He was forced to head out into the wastes with only half a clan; the other half decided to stay and join other Sable tribes. Normally, Sun-Through-Clouds says little in tribal council meetings. He is happy to let Queen Many-Feet make the decisions about the day-to-day running of the tribe, according to the Covenant of Waha. He only interjects when he feels his role as protector is necessary. During the men's meetings, he also speaks little, preferring to allow the younger men to make speeches and put forward alternative courses of action. He allows them to argue until feelings either run too high or the argument is finished. When he does speak, everyone listens as he gives wise words and acts to balance the factions in the tribe, to preserve what little unity there is left. He is expected to die soon. Because of this, and because he has no sons to succeed him, many braves are vying for position to be the next Khan. They are split into two factions: the pro-Lunars, led by Queen Many-Feet (with the brave Hyena-Tail as her spokesman) and the anti-Lunars, led by Little Mouse, the Storm Khan. At the moment, the pro-Lunars are ascendant within the clan as they have persuaded Sun-Through-Clouds to return to Prax. Many braves are allying themselves to one or other of these factions. Sun-Through-Clouds expects the clan to split again when he dies, and he prays that he can survive and keep the clan together until it can regain strength in Prax. === Rani Many-Feet === (Wilful, protective, manipulative) This woman is the herd-mother, comforter, healer and guide of the people. She is skilled in the lore of Sables and can find water and fodder in even trackless desert. However, it is often that the oases are occupied by other, stronger clans. Her sole passion is for the health and safety of the herds and she will endure any hardship and undergo any trial to preserve them. She feels that she is the best able to lead the clan and that its destiny lies back in Prax, no longer an outcast from the Sable Nation. She has persuaded Sun-Through-Clouds to lead the clan back there and is encouraging support within the clan for her plan to join the Lunar sympathisers. === Nine Sticks, Speaker-to-Spirits === (Deranged, Delusional, Impertinent) This shaman is the clan's speaker to spirits and its ancestors. He deliberately sets himself apart from the remainder of the clan, preferring the company of the inhabitants of the Other Side to his own kind. To show his separation, he dresses in the clothes of a woman. He fought at Moonbroth, and only survived the Lunar treachery because he know the secrets of Inora, the White Princess. === Limping Bull, Storm Bull Berserk === (Fanatical, arrogant, violent) He leads the Running Dogs society, open to only those men that have killed Chaos; there are three members at the moment. This society also acts as 'police' in the clan. There are moves to expand the membership of this society, following the massacre of the clan's Storm Bullers. === Bat's Eye === (Eloquent, companiable, scathing) This ageing brave acts as Khan Sun-through-Clouds' eyes and ears throughout the clan. People don't mind this, as he is always friendly and his tipi is always filled with people eager to hear is stories of far-away lands and strange people. He often uses these occasions to feel the mood of the men, and uses his quick tongue to chastise those that have not acted like a true warrior. Due to his way with words, he is also the clan's herald. He is the one that goes out to meet and parley with other clans when we meet. === Hyena-Tail === (Weak-willed, frustrated, eloquent) This brave is the mouthpiece of Queen Many-Feet in the men's council. He is despised for being so weak-willed as to be lead by a woman, but many people agree with what he says. He is not a great warrior, but is a competent hunter and has captured some enemies' beasts. == Distinguishing Clan Features == * ''Animals'': yellow-stained tail (the yellow die is obtained from boiling down Sable gall-bladders). * ''Men'': yellow feathers in headband, zigzag tattoos on cheeks. * ''Women'': twice-braided hair, yellow stripe down the back of tunics. == Spirits worshipped == {| ! Spirit !! Highest ranking member !! Other high-lvl !! Other initiates !! Lay members !! Temple size |- |Waha || Sun-Through-Clouds (Khan) || 1 || 15 || 5 || Small |- |Erithra || Many Feet (Rani) || 2 || 24 || 10 || Small |- |Daka Fal || Nine Sticks (Shaman Priest) || 1 || 1 || 3 || Shrine |- |Storm Bull || Limping Bull (Initiate) || 0 || 0 || 2 || None* |- |Foundchild || Fallen Branch (Initiate) || 0 || 4 || 5 || Shrine |- |Chalana || Flowers' Scent (Acolyte) || 0 || 2 || 4 || Shrine |- |Issaries || Bat's Eye (Acolyte) || 0 || 1 || 2 || None |- |Lankoring || Small Hawk (Lay member) || 0 || 0 || 0 || None |- |Humakt || Antak-Dor-Tak (?) || 0 || 0 || 1 || None |- |Eurmal || Trail's End (Initiate) || 0 || 0 || 13 || None |} * The shrine to Storm Bull is temproarily inactive, due to a near-zero congregation. == Demographics of the Yellowtail Clan == {| ! Ages !! Male !! Female !! Total |- |0-1 || 1 || 1 || 2 |- |1-5 || 3 || 3 || 6 |- |5-10 || 3 || 3 || 6 |- |10-15 || 2 || 3 || 5 |- |15-20 || 2 || 3 || 5 |- |20-25 || 2 || 3 || 5 |- |25-30 || 2 || 3 || 5 |- |30-35 || 2 || 2 || 4 |- |35-40 || 1 || 2 || 3 |- |40-45 || 1 || 2 || 3 |- |45-50 || 1 || 1 || 2 |- |50-55 || 1 || 1 || 2 |- |55+ || 1 || 1 || 2 |- |Totals || 22 || 28 || 50 |} Plus 6 female slaves aged 20-30 === Herd size === {| ! !! Current Herd !! Required Herd |- |Herd size || 400 || (560) |- |Males || 40 || (56) |- |Females || 360 || (504) |- |Males born || 61 || (120.20) |- |Females born || 54.1 || (106.60) |- |Male calves dying || 42.7 || (60.10) |- |Female calves dying || 37.9 || (53.30) |- |Male adults dying || 4 || (5.6) |- |Female adults dying || 36 || (50.4) |- |Dead animals (adult equiv) || 80.3 || (112.70) |- |Net herd gain || -5.4 || (57.4) |- |Females giving milk || 208.8 || (226.8) |} You can read the [[history of the Yellowtail Clan]]. [[Category:Yellowtail]] [[Category:Glorantha]] c3e80e3d7387fd76a9e37f7b701dfcc9a118a3fd 0 1488 1939 1631 2009-02-03T22:00:15Z Neil 2 wikitext text/x-wiki A couple of Praxian-themed songs, inspired by the [[Category:Yellowtail|Yellowtail campaign]]. For more Gloranthan filk, see [http://www.etyries.com/songbook/ Nick Brooke's song page] and [http://www.jane-williams.me.uk/glorantha/songs.cfm Jane Williams's song page]. == We Will Raid You == :to the tune of Queen's ''We Will Rock You'' Pentans you're a horde make a big noise<br> Ridin' in the snow<br> Gonna be big trouble some day<br> You got hair on yo' face<br> You big disgrace<br> Ridin' your horses all over the place :We will, we will raid you<br> :We will, we will raid you Pavis you're a hard tent, rock tent<br> Pitched by the river<br> Gonna take over the valley some day<br> You got dust on yo' face<br> You big disgrace<br> Hiding from the trolls eatin' up your place :We will, we will raid you<br> :We will, we will raid you Quivin you're a high place, tall place<br> Hero says he's come,<br> Gonna make you some peace some day<br> Yoy got woad on yo' face<br> You big disgrace<br> Fightin' your feuds all over the place :We will, we will raid you<br> :We will, we will raid you<br> :We will, we will raid you<br> :We will, we will raid you<br> == We're Waha's Champions == :to the tune of Queen's ''We Are the Champions'' I've paid my dues, time after time<br> I've walked in exile, but committed no crime<br> And bad mistakes, I've made a few<br> I've had my share of sand kicked in my face, but I've come through :We're Waha's champions, my friends<br> :And we'll keep on raiding till the end<br> :We're Waha's champions, we're Waha's champions<br> :No time for losers<br> :'Cause we're Waha's champions, of the Plaines I've taken my cheers, and my tribute gifts,<br> You brought me scalps and beasts and everything that goes with it<br> I thank you all<br> But it's been no padded saddle, no Waha trail,<br> I consider it a challenge before the whole Greatland race,<br> and I ain't gonna fail :We're Waha's champions, my friends<br> :And we'll keep on raiding till the end<br> :We're Waha's champions, we're Waha's champions<br> :No time for losers<br> :'Cause we're Waha's champions, of the Plaines :We're Waha's champions, my friends<br> :And we'll keep on raiding till the end<br> :We're Waha's champions, we're Waha's champions<br> :No time for losers<br> :'Cause we're Waha's champions [[Category:Yellowtail]] [[Category:Glorantha]] 5b9e64d44c07976959441061aaa3885e725eaee8 0 1503 1940 1653 2009-02-03T22:00:58Z Neil 2 /* 1613: Starbrow Rebellion */ wikitext text/x-wiki <blockquote>by Terry Harvey-Chadwick</blockquote> == Asrelia's Age, The God Time == In Asrelia's Age gods and people walked the world together, and neither could tell which was which. Our ancestor was the First Ancestor, called Grandfather Life. With Grandmother Life he had hundreds of children who, in turn, had hundreds of their own. The cosmos was at peace, but slowly awareness woke. Our ancestor's awareness awoke when she realised that she was different from other people, for she was among the Treasured People. Our ancestor lived in the beautiful, luxurious vaults and rooms of Asrelia. She grew up with the Rich Swan, one of the Earth Treasures of Asrelia. === The First Ceremony === At first no-one was ever hungry, for Asrelia contained enough food for everyone. But suddenly, some people were! Ernalda stopped the hunger by teaching people how to do the right ceremonies. These allowed people to find her sister, Esrola, who had been trapped behind a terrible mask by Angdertha. This was the first worship, one of the actions that separated mortals from deities. Our ancestor led the Sacrifice Dance, which our women still perform every year in the spring and discovered the secrets of preparing the bounty of Ernalda. == The Age of Stagnation == The Bright Emperor enslaved everyone and demanded absolute obedience that was little more than death-of-the-soul. But there was once force he could not control, and that was freedom. One little freedom became many, and they began to bring bigger freedoms. Finally, the great sky lay upon the fertile earth and begot Umath Ever-changing, the First Storm, who would bring change to the Bright Emporers stagnant realm. === The Umath Roots === Umath was an upstart god at first, with only a few crazy and dangerous brothers at his side. Gradually he gathered gods and men to his band and, with their help, reshaped the world. Our ancestors were one of the first mortals to help Umath when he separated the sky and earth, and was instrumental in this first act of violence. == The Storm Tribe Age == Umath's youngest and wisest son, Orlanth, made the world as it is today. He married Ernalda, formed the Storm Tribe, and established the first laws and customs that made us what we are today. Orlanth is the King of the gods, whom men follow and women desire, and around whom the winds flock. Where the Bright Emperor sought to control people through fear, force and hollow laws, Orlanth gathered his people through force of virtue, character and example. Our ancestors were amongst the first to recognise Orlanth's greatness and followed him from the start. === The Wedding of Orlanth and Ernalda === Orlanth wooed and wed Ernalda. They invited the whole world to attend their nuptials and, since our ancestors were amongst their friends, they attended. When they entered the sacred space, they were asked if they wanted to stand on the right (Orlanth) or the left (Ernalda) side. Our ancestors chose to guard the sacred circle, with Elmal. === The First Ancestor Clans === At Ernaldas urging Orlanth made the Storm Tribe. He recognised the First Ancestor Clans. Grandfather Life was killed, so all of his descendants are doomed to die as well. The First Ancestor Clans recognise the ultimate distinction between gods and humans. Our ancestors joined the Urlanthing clan, the people of Urlanth, as primal member of the Storm Peoples. == The Vingkotling Age == Vingkot was the first human to rule over many clans. The Vingkotlings were great, with full baskets and fat herds, and always victorious over their foes. The people set up many altars, where the gods lived, and so established the relations of worship. King Vingkot led and sent many expeditions against the enemies who wanted to take it away. Vingkot is still worshipped for the great deeds he did, and for his leadership of the Storm Tribe when Orlanth went away. === Ancient Enemies === The foes of the Vingkotlings were many. As time went on we found ourselves fighting the same foes over and over again. One enemy in particular, the Darkness People, or Trolls, plagued our people consistently during this era, eating everything on our lands that they came across. Our ancestors fought these enemies so often we developed special magic to combat them. Even today our clan does not get on very well with the descendants of our Ancient Enemy, and our Wyter is especially effective against them. === The Gods War === The Gods War was long and destructive. When the gods clashed mountain ranges fell, the sky broke, and the earth was torn to shreds, like a rabbit in the jaws of a wolf. Our ancestors fought a lot, for those who did not fight died, and have no descendants today. Legendary heroes fought in those ages. They are many, although some are unknown outside our clan, who tell their tales. Our clan had a friendship with Helamakt, the Great Defender, helping him at the Battle of Ur Ain, where he drove the salty gods of the sea onwards like froth in a storm. Due to this ancient friendship we now maintain a small temple to Helamakt in the High Hills, near the Upper Heights Stead. Our clan wyter is able to draw upon the fighting storm and blow our foes over, making them easier to defeat. === The Three Red Peak Battle === One time among many, Vingkot travelled north to fight Dara Happa. Although he had done this before, this time he only took a few warriors with him, because he wanted to travel secretly. He took the three sons of Orand, and it was good that he did, because each if them saved the warband when they were in danger of failure. Our clan is one of those few whose ancestors participated in this great raid. Umak Redshot was a son of the warrior Orand, who rode in the warband of Kodig Vingkotsson. Umak made a special sacrifice to Orlanth, as shown by the scars cut in all the men of our clan. He used this power when Jagrekriand attacked. He had time for one shot, and it struck the enemy god hard. All of our warriors are able to learn this feat. === Subject Peoples === When the world fell apart, Vingkot welcomed refugees and reorganised the devastated clans, so that all might survive. He asked us to shelter the people of a ruined clan but let us choose what status to give them. We took in the Korsto Viv, the People of the Rivers, and gave them equal rights: the farmers became carls and the warriors became weapon thanes. Because of this, our clan does not agree with the taking of thralls. == The Chaos Age == Eventually, the Great Darkness descended on Glorantha. Many gods died, others slept, and even Orlanth abandoned his people to try to save the world. Everything in the universe seemed hostile, and was at least uncaring. At last, only monsters, demons, formless Chaos, and their victims remained. Many were so terrible that nothing could withstand them. Clans, tribes, kingdoms and nations disappeared in this terrible time. Some things could be resisted with fire, magic and spear, and all people alive today are descended from someone who survived this time. During this time many beings came to us, feigning friendship, only to betray us, so many times that our Wyter has a sensitivity to feigned friendships, and is able to warn us when these false friends arrive. Everyone who survived the Great Darkness did so because of a survival secret they learned. Most were saved by a living god, who protected them until Orlanth and Ernalda returned. Our clan survived by always moving on when danger threatened, by skating away on the ice. Since then all people in our clan have been good at skating on ice. === I Fought We Won === The entire world was on the brink of destruction and it came down to one person to resist Wakboth, the Devil. That person did, and so the world was saved. You learned in the initiation rites of our clan that you were that individual. Heort is the one who teaches you this, and so you are called a Heortling. Heort reorganised the people onto new clans. Many people came together. Although they became one people they had different traditions and ways of governing themselves. Our clan decided to emulate Orlanth and formed a Storm Ring, first established when Orlanth gave out the great God Treasures to his followers. == The Silver Age == King Heort guided our people out of the Great Darkness. He re-instituted sacrifice to the gods, organised people into clans, gave out the new laws for society, and arranged the first new tribe. In his honour we became Heortlings, part of the Koroltes Tribe, which no longer exists. He also arranged alliances with all the nearby Elder Races and formed the Unity Council. The Unity Council of Silver Age Dragon Pass was a unique and extraordinary event. Many ancient feuds and hostilities were set aisde for the sake of harmony. The unity allowed the peoples to cast off Chaos and to thrive in comparison with the rest of the miserable Darkness-plagued world. For the first time ever humans and the Elder Races worked co-operatively. During this time of peace and co-operation our ancestors made alliances with the Mostali, whom we had previously called dwarves. The Mostali taught us the proper gestures and words to make a Mostali greeting, so that they will meet with us if it seems profitable or harmless. == The Hero Age: History == The unity between peoples established during the Silver Age did not last forever. Terrible things happened, and great wars destroyed whole lands and races. Some peoples remembered the old ways to live in the new world, and so they survived. === Lokamayadon === Orlanth always encouraged personal ambition. No-one thought there could be too much personal advancement until a priest called Lokamayadon took upon himself the powers of High Storm. He tried to replace Orlanth, peacefully at first and then by killing all who worshipped God. Most people went along with him because he was powerful, but most Heortlings resisted him, although it cost them greatly. Our ancestors fought the High Storm people when it was safe to do so, and the clan wouldn't be endangered too much. All Heortlings hate Lokamayadon and what he did, and when Lokamayadon's name is mentioned our clan automatically makes the sign of protection from evil gods with our right hands. === Arkat === Lokomo helped create a new god called Gbaji, which brought Chaos back. It would have destroyed the world except that the great hero Harmast Barefoot performed the Lightbringers Quest and brought back the destroyer of Gbaji. Arkat was the destroyer. He was a man who had lived before, a sorcerous demon in the form of a man, or maybe just a trickster. Arkat helped everybody, then betrayed them. He deserted his army when it fought Dara Happa. He left behind Durega, his lover; and he finally even left Harmast in the clutches of the She Shadows, his worst foe, so that he could betray the whole human race and become a troll. Everyone hates Arkat, even though he destroyed Gbaji and saved the world. Our clan suffered terribly at his hands, but we survived, even though we sent all of our warriors to fight with him. === The Empire of Wyrm's Friends === During the Imperial Age we were approached by a dragon, in the shape of a human, but with the soul of a dragon. The dragons had always been hostile or, at the best, utterly uncaring, but now it offered to share knowledge with us. Orlanth has always been the dragon-slayer, but many priests of other clans embraced these new ways and formed the EWF. Our ancestors resisted the dragons, for we would never believe them, although it cost us dear. Since that time we have always had reason to fear dragons and their kin. === The Dragonkill War === The EWF was corrupted by dragon powers and finally destroyed when the Dragonewts destroyed its leaders. Afterwards, several great armies invaded dragon Pass from the north, east and south. After decades of warfare, hundreds of dragons (including the great dragons) appeared, some as big as the sky, and ate every human being they found, hostile or otherwise. A few incredibly lucky people survived, said to number only en humans. Any hatred people felt for the dragons was replaced by unconquerable fear. We learnt that knowledge isn't harmful in itself, only the way it is used. After the Dragonkill a series of Death runes were carved in stone and erected as the Cross Line, beyond which lay a land forever cursed. "Beyond that line is dragon, this side is us. None may cross except to make pain, horror and terrible death." For centuries, no-one even went close to it. Anyone who dared to cross was never heard from again. == Historical Memory == After the Dragonkill our ancestors lived in Volsaxar (Northern Heortland). We were a part of the Volsaxi Tribe, whose kings had no dynasties and operated in the old way, as rings. The clans were independent, always feuding with the Trolls of the Shadow Plateau and Kitori Wilds. Whitewall is a famous stronghold of Volsaxar, and Smithstone is the ancient capital. However, our kings were themselves vassals of Ezkankekko, a troll demigod who lived in a huge volcanic palace on the Shadow Plateau. Ezkankekko ruled with justice and indifference, and generally kept the peace. === The Resettlement === In the year 1313ST a man swan ashore and began to cause trouble in the land of Kethaela. He confronted the Only Old One, Ezkankekko, and killed him. Thereafter, the usurper called himself The Pharaoh. The Pharaoh went throughout Kethaela and established his rules over the native peoples. When he came to our kingdom he used deceitful magic to overthrow our king, and Heortland became part of his kingdom and civil war broke out. At the same time as this was happening we heard that the lands to the north, in forbidden Dragon Pass, were once more open to humans. We decided to explore these new lands, so we planned and, when we were properly equipped and prepared we found some good new land and settled in Dragon Pass. As clans settled they began to fight one another. Some were conquered, some received tribute from dozens of others, and some formed alliances. We joined early into the Cinsina tribe and gained prestige in the tribe. === King Sartar === About 150 years ago a man named Sartar came to our land of warring tribes. His ways were new and strange, but his magic was powerful. He made this place into the kingdom that now bears his name. He gained most admiration from us when he immolated himself and rose into the sky, leaving behind The Flame of Sartar, which continued burning without fuel until the Lunars extinguished it. === The Lunar War === High King Sartar exchanged ambassadors with the king of Tarsh to promote peace and trade. Men and women in red robes came after his apotheosis, and for the first time people in Sartar learned about the Lunar goddess. The Red Moon had risen in 1247, a bad omen that surprised and frightened the Heortlings. Everyone became uncomfortable when they learned that the missionaries worshipped the red thing in the sky, for all the Sartarite prophesies warned that the Red Moon was a foe. Nonetheless, the Red Goddess' missionaries came through Sartar, seeking converts to her way. We gave the missionaries hospitality at this time. However, the Lunar Empire conquered Tarsh many years ago , then they tried to conquer Sartar as well. The great High King Tarkalor led everyone to many years of victory in battles with the Lunar Empire, and we fought honourably, but not insanely, as we also had the overall welfare of the clan to think of. Then there was the terrible defeat at Grizzly Peak, and the Lunars won the war. They occupied Sartar and closed down all the city temples of Orlanth, levied taxes, and enslaved those they deemed criminals or rebels. They imposed many laws that were impossible to obey, just so they had excuses to tax the clans and arrest anyone. We chose to bide our time until a good day for rebellion came. With no Orlanthi king on the throne of Sartar, many of our countrymen took advantage of the opportunity to settle old grudges or steal from neighbours. The tribes fell to fighting one another. However, along with some other clans, we attacked the Telmori, a werewolf race that eats people and has never worshipped our gods. Cattle were given and taxes lifted to all those who gave aid. Last year many clan chieftains and tribal kings, led by the Vingan warrior Kallyr Starbrow, took part in an uprising against the Lunars. We said that it was not the time for rebellion, but some of our warriors fought, even though we forbade them to. When the Lunars conquered Sartar they sent assassins, demons and gods to kill all the heirs of the house of Sartar. However, during Starbrows Rebellion an heir was found in the south, a scholar named Temertain. He is now prepared to be the legendary "Philosopher King". His dynastic blood has been proven, but he could not light the flame of Sartar that the Lunars extinguished in 1602. So far, his philosophy seems to be close collaboration with the Lunars. However, even Lunar supporters know that he is weak, just a Lunar puppet. We hate him, but there is nothing to be done. The Lunars have made the Durulz scapegoats for the Kallyr's rebellion, and put a bounty on their heads. However, due to our ancient friendship with the river, we have never killed a Durulz just to make our lives a bit easier. == Recent Events == === 1607=== The neighbouring Maboder tribe are wiped out by the Telmori. Cinsina warriors join Duke Jomes Wulf in conquering Telmori. The Maboder lands are given to Jomes Wulf and renamed Wulfsland. === 1613: Starbrow Rebellion === The Dinacoli tribe attack Dangerford but are beaten off by Cinsina warriors. The Lunars march on Dangerford and the Decamoli surrender with good terms. Dangerford is now occupied by Lunar garrison, although the Decamoli still live there and collect tolls from river traffic. The Blueberry and Gorde clans are forcibly reassigned to the Cinsina tribe from the Culbrea by the Lunars, after Starbrow's Rebellion is crushed. [[Category:Tertorae]] [[Category:Glorantha]] 30f8c8d4209bbfc134cd623ac2e0de8960b41c17 0 1500 1941 1652 2009-02-03T22:01:22Z Neil 2 /* Joral Trollkiller: The Clan Wyter */ wikitext text/x-wiki <blockquote>by Terry Harvey-Chadwick, based on official material from [http://www.glorantha.com Issaries Inc.]</blockquote> {| cellpadding=3 align=right width=50% border=1 | === Clan Statistics === * '''Lunar Friendship Rating:''' -1 * '''Wealth:''' 5W2 * '''Keywords:''' ''Women:'' +4 Cooking; ''Men:'' +3 Initiate or Devotee of Orlanth; ''All:'' Skating +4 * '''Attitudes:''' Fear Chaos 17, Fear Dragons 15, Friendly to Volsaxings 17, Hate Arkat 1W, Hate Lunars 14, Hate Trolls 3W * '''Clan Secret:''' Hit Jagrekriand with Missile (+10 to Ranged Combat skill once per contest, when appropriate) * '''Motto:''' "Violence is always an option" |} The Tertorae clan are part of the Cinsina tribe, also known as the Brave Tribe, but called the Wolfslayers amongst themselves. Their lands encompass the area enclosed by the Solfint and Lorthing Rivers to the north and west, Bragi's Ford and the High Hills to the south, and part of the lands between the Vinga's Throw Hills and the Bull Hills to the East. __TOC__ The Tertorae are a traditional balanced Orlanthi farming, hunting and herding clan, although a large minority worship Elmal. [[Image:Tertorae-tula-small.jpg|left|thumb|200px|Tertorae clan tula]] The main stead is Wulfsstead in a deep valley of the Thunderstone Heights, as it is well protected and central to most of the major places of importance. Other important steads include Founders Stead, Broken Fang Stead, Lightstead, Upper Heights Stead, and Harststead. The clan consists of approximately 1400 adults, making it the most numerous clan of the Cinsina. == Political Affiliations == The Tertorae tula holds the largest Cinsina Ernalda temple, a small Vinga temple, and a shrine to Helamakt. The Tertorae are allied with the Decamoli and Drutorae clans. Relationships with the Halfari clan are strained as Ivar Quickstep, chieftain of the Halfari, also claims the Tribal Kingship. Tertorae warriors often raid into the Greenhaft, Barlamani and Marthiording clans of the Culbrea tribe. They have also recently started raiding the Gwandor. Raiding of the Blueberry and Gorde clans stopped when they were forced into the Cinsina tribe by the Lunars. Relations with Wulfsland are cordial. The current clan Chieftain and Tribal King is Ivartha the Skinner, a Vingan chosen to be politically neutral to the Lunars. == The Tertorae Clan Ring == === Lightbringer Ring === ;Orlanth :Ivartha the Skinner (Vinga & Orendana) ;Issaries :Jonrik Goodvoice (Harst) ;Lhankor Mhy :Enfrew Sharptongue (Andrin) ;Chalanna Arroy :Sora Gentletouch (Jera) ;Eurmal :Orlkarl Crookleg ;Flesh Man :Hengall Venharlson (Durev) ;Elmal :Ranulf the Whisperer (Beren) === Ginna Jar === Vorana Ericsdottir (Harst), Huma Lightbreeze (Ormalaya), Bavari Torkilson (Elmal), Ketil the Axe (Orlanthcarl), Arkell Brightspear (Elmal), Bragi Stonehead (Voriof), Kort Thundersling (Hedkoranth), Yrsa the Shrill (Vela Matchmaker), Erica Longhair (Orane Steadwife), Torkil the Stout (Poverri), Astrid Honeydew (Minlinster), Olaf the Lofty (Ohorlanth), Angantir Longarm (Harst), Janerra Keeneye (Manhome), Kestra Fleetfoot (Vinga), Meera Cowsong (Uralda), Morag Blue-eye (Esra) === Other Important People === ;Clan Champion :Pallashee LongAxe (Kargan) ;Warband Leader :Halgrim Sharpsword (Starkval) ;Master Hunter :Korst the Tracker (Odayla) == Joral Trollkiller: The Clan Wyter == {| cellpadding=3 align=right width=50% border=1 | === Wyter Statistics === * '''Physical Manifestation:''' Ancient Bronze Spear * '''Communication Manifestation:''' All spearheads hum. * '''Awareness''' 3W3: Sense Trolls; Sense False Friends * '''Defense''' 17W2: Ignore Restraints; Defend Against Trolls * '''Blessings''' 19W2: Sacrifice Dance +5; Thunder Blessing; Blow Down Foe; Skate Marvels; Dwarf Friend |} Joral was a warrior of Helamakt during the Vingkotling age. During that time his clan was beset by trolls, who ate the crops, the wild woods, the cattle and other domestic animals, and many of our ancestors. Joral defended his tula with the powers of the wind and rain, blowing down their ranks and slaying them with his spear. The trolls feigned peace, captured him and bound him in lead chains. Joral escaped and fled over the ice on skates made from the shinbones of the troll guards who tried to stop him. Joral was then captured by dwarves, but convinced them of his friendship, and they aided him against the trolls. In the Greater Darkness, Joral's clan gave hospitality to many different people, outcasts and refugees. Several times a group of guests planned to betray their hosts, but Joral discovered their plotting and showed them for what they truly were. Joral was slain by the great troll champion Gork, but Joral's spear was used by another warrior to avenge him. Joral agreed to become the clan Wyter and look after his descendants forever more after his death. Since then he has protected us well, even warning us when the Lunars came with the false promises, so we do not trust them. [[Category:Tertorae]] [[Category:Glorantha]] 6a824cd311337340f74aecc6a0ca7cc74f59eb10 0 1504 1942 1655 2009-02-03T22:01:47Z Neil 2 /* Steads */ wikitext text/x-wiki <blockquote>by Terry Harvey-Chadwick</blockquote> == Bloodlines == [[Image:Tertorae-tula-small.jpg|left|thumb|200px|Tertorae clan tula]] There are four bloodlines in the Tertorae: ;Emanari :Descended from Emanara, a daughter of Queen Cinsina. By far the most numerous bloodline, including the current Tribal King, Ivartha the Skinner, amongst its members. ;Korting :Descended from Kort the Elder, a priest of Ohorlanth the Great Storm, and a prolific heroquestor. ;Harivari :Descended from Harivig the Red, a weaponthane of Queen Cinsina. This bloodline tends towards Elmal worship, and breed fine horses. ;Bragii :Descended from Bragi Closemouth, a thane of Emanara. == Steads == {| cellpadding=3 ! Stead !! Bloodline(s) !! Adults !! Notes !! Household Head |- | Founders Stead || Emanari || 74 || Oldest stead. Market here. || Vorana Ericsdottir |- | Wulfsstead || Korting / Emanari || 80/21 || Chieftain's Stead || Enfrew Sharptongue* |- | Lightstead || Harivari || 87 || Mainly worship Elmal. Breed horses || Ranulf the Whisperer* |- | Hunters Lodge || None || - || Used by hunters. No permanent residents || Huma Lightbreeze |- | Woods Stead || Harivari || 58 || || Bavari Torkilson |- | Ketilsstead (in woods) || Harivari || 47 || || Ketil the Axe |- | Ormson Stead || Harivari || 62 || || Arkell Brightspear |- | Vingasstead || Emanari || 70 || Produced many redheads || Ivartha the Skinner* |- | Lost Sheep Stead|| Emanari || 77 || Herd Sheep || Bragi Stonehead |- | Kortstead || Korting || 73 || Founded by Kort || Kort Thundersling |- | Varnisstead ||`Korting || 62 || || Yrsa the Shrill |- | Crown Stead || Emanari || 91 || || Erica Longhair |- | Fishers Stead || Bragii || 53 || Mainly fishermen || Torkil the Stout |- | Ketilstead || Korting / Emanari || 39/48 || Known for their mead || Astrid Honeydew |- | Harststead || Emanari || 81 || || Jonrik Goodvoice* |- | Upper Heights Stead || Korting || 43 || || Olaf the Lofty |- | Bragistead || Bragii || 69 || Founded by Bragi || Hengall Venharlson* |- | Broken Fang Stead || Emanari || 77 || || Angantir Longarm |- | Deer Stead || Bragii || 62 || || Janerra Keeneye |- | Ernalda's Temple || Emanari || 79 || Main Cinsina temple || Sora Gentletouch* |- | Vinga Temple || Emanari || 39 || Founded by Emanara || Kestra Fleetfoot |} * Member of the Clan Ring [[Category:Glorantha]] [[Category:Tertorae]] 9d8ce493fcdcd47a6780b7577517a512af3c3387 0 1538 1949 1948 2009-02-09T17:31:34Z Neil 2 /* Bare bones deployment: Apache2 and mod_ruby */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production.* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 0e50c1bd433f054cbd77ef4c1f1dc0a59be2f47a 1950 1949 2009-02-09T17:33:56Z Neil 2 /* Bare bones deployment: Apache2 and mod_ruby */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <tt>gem</tt> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <tt>gem</tt> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <tt>mongrel_cluster</tt> on the desktop machine.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production.* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 741139ace74fa83cf4735d3e3a3a2e478c55f4db 1951 1950 2009-02-11T00:15:08Z Neil 2 /* Installation of Rails */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install capistrano root@server:~# gem install mongrel_cluster :(Don't need <code>mongrel_cluster</code> on the desktop machine; don't need <code>capistrano</code> on the server.) :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production.* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 4cd204307d94f487be1d705fc7a84096f8bd3be9 1953 1951 2009-02-16T15:11:08Z Neil 2 /* Installation of Rails */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production.* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions]. * Capify the project: user@desktop:~/project# capify . * Modify <tt>project/deploy.rb</tt>: set :application, "project.domain.tld" set :repository, "http://svnsite.doman.tld/svn/repo1/project/trunk" set :deploy_to, "/var/www/#{application}" role :app, "server.domain.tld" role :web, "server.domain.tld" role :db, "server.domain.tld", :primary => true * On the server, create the <tt>/etc/apache2/sites-available/project.domain.tld</tt> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/project.domain.tld/current/public> Options FollowSymLinks AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Make the <tt>/var/www/project.domain.tld</tt> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown /var/www/project.domain.tld root@server:~# chmod a+rwx /var/www/project.domain.tld/ * Create the database root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Set up and check the deployment user@desktop:~/project# cap deploy:setup user@desktop:~/project# cap deploy:check :Fix any reported errors * Make the first deployment user@desktop:~/project# cap deploy:cold 021ebffdf42ba9245c7e472dcd07dfe006dd144e 1954 1953 2009-02-16T17:19:47Z Neil 2 /* Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production.* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations afe4bd386a453cdac309e9ea046835210f4f047f 1955 1954 2009-02-16T17:26:34Z Neil 2 /* Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project' identified by 'password'; mysql> grant all on project_production.* to 'project'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations 66537fa7b986041cb774e1ca4163d24fbbfa4cbd 1956 1955 2009-02-20T14:19:26Z Neil 2 wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations 235b3e65ea33f6cca35cdd2406f48006bb22d475 1957 1956 2009-02-21T22:35:36Z Neil 2 /* Installation of Rails */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations 05e31256a0e23f3cef15a74bd6a859eca2a1556b 1958 1957 2009-02-21T22:42:44Z Neil 2 /* Installation of Rails */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations 58d57f3824d06f6f8bc1cf9f4904995ebf101bdf 1960 1958 2009-02-21T22:59:15Z Neil 2 wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == See also == * [http://wiki.rubyonrails.org/rails/pages/RailsOnUbuntu Rails on Ubuntu howto] * [http://github.com/peter/rails-on-ubuntu/tree/master Another worked example of setting up Rails] == Ruby and Rails idioms == * [http://wiki.rubyonrails.com/rails Ruby on Rails wiki] and [http://wiki.rubyonrails.org/rails/pages/Howtos Howtos] * [http://wiki.rubyonrails.org/rails/pages/RailsBestPractices Ruby/Rails best practices] * [http://www.rubyinside.com/21-ruby-tricks-902.html 21 Ruby tricks] (use with caution) c319b8882ca66d0b4b04d5c697a5611ee4dbe65f 1972 1960 2009-02-27T10:37:48Z Neil 2 /* See also */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations === Subsequent deployments from a stable branch === Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/rails/pages/RailsOnUbuntu Rails on Ubuntu howto] * [http://github.com/peter/rails-on-ubuntu/tree/master Another worked example of setting up Rails] == Ruby and Rails idioms == * [http://wiki.rubyonrails.com/rails Ruby on Rails wiki] and [http://wiki.rubyonrails.org/rails/pages/Howtos Howtos] * [http://wiki.rubyonrails.org/rails/pages/RailsBestPractices Ruby/Rails best practices] * [http://www.rubyinside.com/21-ruby-tricks-902.html 21 Ruby tricks] (use with caution) e11f160d2053ad5ed6293104a50178e2a156d8c2 1973 1972 2009-03-02T19:24:26Z Neil 2 /* Database creation */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations === Subsequent deployments from a stable branch === Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/rails/pages/RailsOnUbuntu Rails on Ubuntu howto] * [http://github.com/peter/rails-on-ubuntu/tree/master Another worked example of setting up Rails] == Ruby and Rails idioms == * [http://wiki.rubyonrails.com/rails Ruby on Rails wiki] and [http://wiki.rubyonrails.org/rails/pages/Howtos Howtos] * [http://wiki.rubyonrails.org/rails/pages/RailsBestPractices Ruby/Rails best practices] * [http://www.rubyinside.com/21-ruby-tricks-902.html 21 Ruby tricks] (use with caution) 69b27b9094740b8613328d347b70626a1008f4ac 1974 1973 2009-03-02T19:25:14Z Neil 2 /* Installing Rails with NetBeans 6.5 on Windows */ wikitext text/x-wiki == Installation of Rails == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == Application deployment == === Bare bones deployment: Apache2 and mod_ruby === [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload === Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger === [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations === Subsequent deployments from a stable branch === Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/rails/pages/RailsOnUbuntu Rails on Ubuntu howto] * [http://github.com/peter/rails-on-ubuntu/tree/master Another worked example of setting up Rails] == Ruby and Rails idioms == * [http://wiki.rubyonrails.com/rails Ruby on Rails wiki] and [http://wiki.rubyonrails.org/rails/pages/Howtos Howtos] * [http://wiki.rubyonrails.org/rails/pages/RailsBestPractices Ruby/Rails best practices] * [http://www.rubyinside.com/21-ruby-tricks-902.html 21 Ruby tricks] (use with caution) cc5f084d149aa8fec44d01e6f4a0942821b41f4f 1981 1974 2009-03-04T10:21:09Z Neil 2 wikitext text/x-wiki See the separate pages * [[Ruby on Rails installation]] to install RoR * [[Ruby on Rails project creation]] for tips when creating a new project * [[Ruby on Rails deployment]] for deploying your Rails app on a server == See also == * [http://wiki.rubyonrails.org/rails/pages/RailsOnUbuntu Rails on Ubuntu howto] * [http://github.com/peter/rails-on-ubuntu/tree/master Another worked example of setting up Rails] == Ruby and Rails idioms == * [http://wiki.rubyonrails.com/rails Ruby on Rails wiki] and [http://wiki.rubyonrails.org/rails/pages/Howtos Howtos] * [http://wiki.rubyonrails.org/rails/pages/RailsBestPractices Ruby/Rails best practices] * [http://www.rubyinside.com/21-ruby-tricks-902.html 21 Ruby tricks] (use with caution) [[Category:Ruby]] [[Category:Rails]] 6eed3aaad9ebe564779fcd67fcd88889bc1f55a2 1995 1981 2009-03-04T13:15:58Z Neil 2 wikitext text/x-wiki See the separate pages * [[Ruby on Rails installation]] to install RoR * [[Ruby on Rails project creation]] for tips when creating a new project * [[Ruby on Rails deployment]] for deploying your Rails app on a server (For more general documentation on server setups, see the [[Server setup]] pages.) == See also == * [http://wiki.rubyonrails.org/rails/pages/RailsOnUbuntu Rails on Ubuntu howto] * [http://github.com/peter/rails-on-ubuntu/tree/master Another worked example of setting up Rails] == Ruby and Rails idioms == * [http://wiki.rubyonrails.com/rails Ruby on Rails wiki] and [http://wiki.rubyonrails.org/rails/pages/Howtos Howtos] * [http://wiki.rubyonrails.org/rails/pages/RailsBestPractices Ruby/Rails best practices] * [http://www.rubyinside.com/21-ruby-tricks-902.html 21 Ruby tricks] (use with caution) [[Category:Ruby]] [[Category:Rails]] 9b58d4f216c084932d96dfacd936550c160e8b3f 0 1569 1952 2009-02-12T20:49:14Z Neil 2 New page: This scenario was written some time ago, perhaps for possible inclusion in the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word] fanzine. However, it quickly grew too large for ... wikitext text/x-wiki This scenario was written some time ago, perhaps for possible inclusion in the [http://www.celtic-webs.com/theunspokenword/ Unspoken Word] fanzine. However, it quickly grew too large for inclusion there. It then languished on my hard disk for a while until I finally decided to post the outline of the scenario here. == Benchmarks == This episode is designed for characters whose best ability is around 10W. It involves physical combat, subterfuge, and ritual magic. Most or all of the heroes should be able to speak Kralori. == Summary == The heroes travel to a small village in the western Shan Shan mountains to investigate rumours of massacres. They find the village deserted, but ally with some neighbouring deer hsunchen. They track down a group of Tong enforcers only to find that they are not responsible for the attacks. The heroes learn that a renegade Huan To is behind everything. Realising the Huan To's plans to invade Kralorela with a huge ghoul army are about to come to fruition, they devise a ritual to defeat him and infiltrate his fortress. The heroes are captured, escape, avoid hordes of ghouls, and break into the Huan To's inner sanctum. They chase the Huan To though the catacombs below the fortress while they debate the nature of good and evil. He is eventually defeated and the heroes destroy his main magical focus, causing the ghouls to die and the fortress to collapse. The invasion thwarted, the heroes return to Kralorela and receive their reward. == Backstory == The massacres are caused by Shia Shan Gua, a Huan To trying to build up an army of ghouls to attack Kralorela. However, his hatred of Kralorela has overtaken him and he is acting before he is ready. He has murdered his master and stolen a powerful magical artefact from his sponsoring Tong, and is using it to control and stabilise the ghouls. He need only complete one last ritual and he can launch his attack. He is, however, having second thoughts, and the Tong want to extract revenge. {| align="center" {{Prettytable}} |{{Hl2}} | === Designer Notes or It's Not Really That Linear === |- |Although this scenario is presented in a very linear fashion, it retains a great deal of flexibility to accommodate all those unusual plans that player heroes come up with. Acts I and III are fairly linear: Act I because it simply gets the heroes to the scenario proper in Plot Turn I, and Act III because, starting from Plot Turn II, the heroes know what to do, and just need to do it. On the other hand, Act II has very flexible structure. The only fairly fixed point is the Midpoint, which marks the time at which the heroes stop being reactive and become proactive. It also represents the false goal that the heroes are initially working towards. The summaries in the other scenes show the purpose of each scene, and those scenes should be used as and when dramatically appropriate. |} == Act I: Introduction == === Scene 1: Prologue (Cut Scene): Shinuan Village === ''Summary: Ghouls attack and wipe out a village in the Shan-Shan mountains.'' Give the players new characters, who are ex-pat Kralori villagers: the Headman, a Farmer or two, Housewives, cute children. Have them working in the fields, tending the cooking, doing chores, etc. Don't let them know the name of the village. Then, a large body of ghouls will appear from the surrounding rocky mountain slopes and attack the village. Describe the ghouls only by appearance (obviously dead, like the best "Living Dead" zombies). Have the players try to defend the village against the ghouls, and perhaps destroy a couple. Have villagers attacked by ghoul spirits, be possessed, and then attack their former neighbours. Eventually, many villagers will be dead, and the rest corralled in some paddy fields or pigpens. Then, a palanquin will be carried into the village and all the ghouls turn to face it, then kow-tow before it. A clawed hand will appear on the door frame, and the door swings slowly open... === Scene 2: The introduction === ''Summary: The heroes are summoned by a mandarin, and given an offer they can't refuse. '' The contents of this scene will vary, depending on who the Heroes are. The Kralori authorities will need to have some hold over the Heroes to persuade them to take the mission, in addition to the rewards promised. If the Heroes are traders already in Kralorela, have them arrested by Kralori police on some trumped-up charge and placed in gaol. Then a Mandarin will come into their cell, and make them an offer they can't refuse. If the Heroes are Kralori police or military, they will be summoned to see their superior, who will say they have been selected for a special, secret mission. They will then make their way to a large town in the foothills of the Shan-Shan mountains, near the Iron Forts. {| align="center" {{Prettytable}} |{{Hl2}} colspan=2| === Interleaving the Prologue and Scene 1 === |- |colspan=2|The dramatic tension can be increased by cutting between the Prologue and Scene 1. Cut between the action in the two places to highlight the similarities between the two scenes. Done properly, your players should be scared for their lives about meeting the mandarin. |- |{{Hl2}}| The Villagers |{{Hl2}}| The Heroes |- |The villagers wake up for another ordinary day. |The heroes wake up for another ordinary day. |- |The villagers head out to the fields and start to work |The heroes open their market stall, or whatever is a normal day for them. |- |There is some commotion in the village as intruders are spotted. |There is some commotion in the market as soldiers barge their way through. |- |The first ghouls leap from cover, howl, and rip the guts from a villager. |The soldiers approach the heroes and inform them they are under arrest. |- |The villagers flee back to their village, pursued by ghouls. |The heroes are escorted to the inner citadel. |- |Death and mayhem as the ghouls stream through the village, ripping villagers apart as they go. |The heroes enter the citadel. Describe the forlorn looks they receive from ordinary Kralori, and the wretched state of the prisoners in the citadel. |- |The surviving villagers are rounded up into a makeshift corral. |The heroes are pushed into a cell. |- |After a while, the villagers watch as the palanquin is carried towards them. "It's the master!" hisses one of the ghouls. |After a while, guards collect the heroes from their cell. "The Mandarin will see you now," says one of the guards. |- |A clawed hand opens the palanquin door. One of the villagers screams in terror as they see what lies inside. |The heroes are ushered into a plush room. Seated at the end is a man in fine robes. Most distinctive about him are his long, talon-like fingernails. |- |Villagers are strong-armed towards the palanquin, screaming, "No! For the sake of my eternal soul, no!" |The mandarin gestures to the heroes. "Please be seated," he says. |- |colspan=2|However, if you don't think your players would appreciate this technique, don't use the Prologue scene. Instead, allow them time at the start of Scene 3 to discover what happened in the village, before getting them embroiled in the fight. |} The tone of interview with the Mandarin, Li Xian, will depend on the impression made by the heroes (a Group Simple contest). '''Make a good impression (14)''': ''Kralori customs, Eastern Customs (-3), Hero's Own Customs (-6) [Master Li knows enough of foreigners' ways to appreciate the correct sentiment], Complementary (-5).'' All contests in this scene, including making a good impression, are subject to carryover, which reflects the general tone of the interview. If the heroes make a good impression, Master Li will stress the benefits of his 'proposal'; if they do not, he will stress the penalties he can impose. Master Li will explain the situation: there have been rumours of Tong activity in the western Shan-Shan mountains. If true, there could be a danger to the Kralori nation, which must be addressed. The PCs are to discover the truth of this. If they accept, they will be richly rewarded. If they refuse, they will be persecuted in Kralorela. The precise nature of this persecution will vary, depending on the type of PCs involved. It could be confiscation of goods, taking of hostages, banning from Kralori lands, official disapproval of trading contacts, etc. Master Li suggests they start their investigations in the village of Shinuan on the area in question. There they will meet an associate of his, Naiman Qi, who can provide them with more information. Master Li will give the heroes a letter of introduction to Master Naiman, and letters of passage to the Talon commanding the Iron Forts, instructing him to give the heroes appropriate assistance. Heroes being heroes, this won't be enough to satisfy them: they will try to get more from Master Li. '''Get a fuller explanation (2W2)''' or '''Be allocated more assistance (15W)''' or '''Increase the reward offered (2W)''': ''Persuade, Debate, Kralori Customs (-2), Bargain (-5).'' Improvise the results of these contests. Try to keep the results of these contests small, as this is not the focus of the episode. Stress how Master Li's mood changes in response to the changes in carryover. If the heroes get too uppity, Li will resist their actions with his Will of Iron. ;Li Xian, Kralori mandarin, member of the Coiled Dragon (Kralori secret service). :Appear Affable 2W, Cold Look 18, Will of Iron 2W2, Condescend 10W, Expect Obedience 10W, Kralori Customs 12W, Foreign Customs 5W2, Darudic Magic 15W. === Scene 2: Travel to Shinuan === ''Summary: The heroes travel to the village of Shinuan.'' {| align="center" {{Prettytable}} |Don't dwell too much on the travel. The purpose of this scene is to get the heroes to Shinuan, where the fun can start. If the Heroes are in eastern Kralorela, simply describe the endless miles of paddy fields seen from their coach and the spartan inns they stay at. They are under discreet but competent guard the whole time. However, this time can be an opportunity for subplots, if your players are interested. |} Wherever they start, the heroes will eventually arrive at one of the Iron Forts in the Broken Dragon pass in the Shan Shan mountains. As they approach the area, the land becomes less fertile with fewer and smaller farms; walls around them become sturdier. As they approach the fort, they feel a disturbance in the magical energies surrounding them. Any active magics falter before failing entirely. Theists will notice an emptiness in their being as their connection with the God Plane becomes attenuated. Animists will feel the pain experienced by their spirits and sense them retreat into their fetishes. Colours and sounds dull as the life-force of the world is suppressed. Magic becomes more difficult, rising to D + 80(*) as they enter the Fort itself. Any magic items that come into direct contact with the iron will crack or split and become forever useless. The heroes' escort will give passwords to gain entry to the fort. The whole fort, passages, walls, and floors, is made from a single piece of iron. They pass through a series of gates and courtyards into the centre of the fort. Courteous but uncommunicative young guards escort the heroes to a small guest house just off the courtyard of the fort, and remain outside the door. All the interior furniture is made from iron, seamlessly joined to the floors or walls. A few cushions and blankets are the only concessions to comfort. A simple meal is brought, and the heroes can watch the bustle of patrols entering and leaving the fort. After a few hours, a junior Talon will arrive and explain that the General commanding the fort has agreed to provide the heroes with native guides to Shinuan, and they are to leave immediately. There is one guide per three heroes waiting outside the guest house. The guides, of the Lo-Fak yak people, shoulder the heroes' equipment and the party leaves the fort. It takes three days to walk across through the barren, rocky, and steep mountainsides to Shinuan. The going is rough, the footing loose, the water scarce, they days hot, and the nights cold. And throughout all this, the heroes have the suspicion that things are slightly more difficult than they need be. <pencil> This region of the Shan Shan is similar to the mountainous regions of Afghanistan and the Hindu Kush. The guides have stone and wood tools and are dressed in leather and furs, and are a sort of cross between cavemen and sherpas. ;Native Lo-Fak guide :Carry Heavy Load 2W, Shan Shan Geography 8W, Forage 20, Scramble Over Rocks 18, Sneak 5W, Make Heroes' Lives Difficult In Small Ways 17, Enigmatic 18, Close Combat 18 (Spear ^3, Flint Knife ^1). === Scene 3 (Plot Turn I): Battle at Shinuan === ''Summary: The heroes explore Shinuan, are attacked from all sides, and eventually make an alliance with some neighbouring Dimali.'' ;Setting Shinuan :The village of Shinuan is in the lower valley of one of the many rivers that flows down from the Shan Shan. It housed around 80 people in a number of communal houses made from the plentiful stones around, with heavy thatched roofs. The village is compact and protected by a dry stone wall the height of a man. The surrounding woodlands have been cleared for some fields for rice and other crops, and pens for pigs and buffalo. The river has been dammed to make fish ponds. In the mid-morning, the heroes approach Shinuan, the only habitation they have seen since leaving the Iron Fort. But things are not as they seem. '''Notice something odd about Shinuan''' (14): ''Farming, Household Management, Herding (-5), Observe (-5).'' The greater the level of success, the earlier the heroes will realise that the village is deserted. There are no workers in the fields, no fires in the houses; no activity at all. If the heroes fail this test, they will not notice anything amiss until they enter the village. The heroes will want to investigate. What happened in Shinuan is described in the prologue. If you played through this, mention it to the players now. If you didn't, you may want to use flashbacks to tell the story as they put the pieces together. An Ability Test of Tracking or similar ability will reveal a large number of human tracks heading up the side of the valley. No human bodies will be found anywhere. Before the heroes have a chance to do anything about it, they are attacked by a group of Dimali warriors, who think the heroes (and the Lo-Fak) are responsible for the 'massacre'. . '''Be aware of impending attack''' (5W, 18 if lookouts are on watch): ''Spot Danger, Scan Terrain (-3).'' If the heroes win, they can organise themselves and perhaps attack or parley with the Dimali before they enter the village, or even leave the forest. If the Dimali win, they will have scaled the walls before the heroes notice them. If the heroes suffer a Major or Complete defeat, they won't notice anything until they receive the first arrow in the back. After a few rounds of fun and mayhem, the Tong enforcers will attack, focussing on the heroes, as they think the heroes are working for Shia Shan Gua. Allow confusion to reign for a while before the heroes can question people from the two groups and sort out what's going on. It shouldn't take them long to discover, and convince the Dimali, that they have a common enemy in the Tong enforcers (take a note of the approach used, and the success of, these attempts: it will colour how the heroes are received in Asquat). Once the enforcers realise the fight is going against them, they will change their objective to getting out of the village, leaving no-one behind (cheat outrageously to ensure this if you have to). After the fight, the heroes retire to Asquat, the Dimali village, for rest and healing. The Lo-Fak guides should all be killed or incapacitated during this fight, perhaps by Dimali with a grudge. ;Things that should happen during the fight * Someone on top of the surrounding wall should knock off someone else trying to climb over it. And someone else on the wall (or even the same person) should have their feet swept out from under them, knocking them off. * Double-takes by the Dimali and Tong enforcers as, while fighting the heroes, they realise they may be fighting the wrong people. * A hero chases a Dimali warrior down an alleyway, only to find a group of three Tong enforcers, who are attacking the Dimali and then start on the hero. * A Dimali seemingly coming from out of a wall after using a concealment spirit to hide himself Predator-style. * An enforcer back-flipping away from the sword stroke that would have gutted him. ;Dimali Warrior (one per hero) :Leader: Tokmak ;Tong Enforcer (three per two heroes) == Act II: Confrontation == === Scene 1: Council at Asquat === ''Summary: The heroes return to Asquat and the village elders of their bona fides, and discuss what needs to happen next.'' ;Setting - Asquat :Asquat is a small village about half a day's walk from Shinuan up the valley. It is a small village of 65 people and consists of a number of small, roughly built stone huts around a central clearing containing a large tree, from which hang many fetishes. Interspersed between the huts are small gardens where root vegetables, squashes, and beans are grown. There is no boundary to the village; the outlying huts are scattered into the surrounding scrubby woodlands. Typical Dimali names are Oytal, Ataloo, Kamur, Ezik, and Ashu. Tokmak and his warriors escort the heroes up the valley towards Asquat, keeping a discreet guard over them. They follow the river through a mixture of open grassland, scrub, and occasional forests. As they climb, the ground becomes steeper and more barren. En route, the heroes are likely to ask the Dimali what they were doing in Shinuan: '''Get along with the Dimali warriors''' (15, modified by events in Shinuan): ''Friendly, Storytelling, Boast (-3), Debate (-5), Hunting abilities (-8).'' Tokmak appreciates tales of derring-do, and disapproves of less active individuals. Failure means the Dimali distrust these foreigners. Success increases the level of trust towards the heroes, and reveals more of the story. There have been rumours of increased Tong activity in the area recently, so the Asquat villagers have been on guard. Some hunters were in the area of Shinuan yesterday and noticed the strange tracks, so they gathered a force to investigate the village. When they saw the heroes, the Dimali assumed they were members of a Tong. Just before nightfall, the party arrives in Asquat. The villagers view the heroes with some surprise: the warriors left to investigate, and have brought back strange foreigners. A great hubbub ensues, and the heroes are quickly brought toward the village elders to account for themselves. Tokmak will tell the village of what happened in Shinuan, and what he thinks of the heroes. This speech will be in the native Dimali tongue, so the heroes will be probably not understand it, though they can can use appropriate abilities to gauge how well they are being received. When Tokmak has finished, Karakol, the village headman, will ask the heroes, in Kralori, to explain themselves and what they are doing in the area. '''Persuade the village elders of their bona fides''' (18, modified by 'carryover' from Tokmak's speech, 64AP, defensive ^4): ''Persuade, Storytelling, Debate (-3), Dimali customs (-5), Convince Buyer (-5). '' This contest could be run as an extended contest. There are four elders (Karakol plus three 'followers'). The village elders' defensive edge is due to the villagers (and probably the heroes) speaking in their second language. This edge can be countered by the heroes augmenting their abilities with appropriate communication abilities (such as Speak With Hands or Interpret Spoken Language). If the heroes fail in this contest, they will remain under guard in the village before being told to leave. A complete failure will mean the Dimali will attempt to murder the heroes in their sleep. A Marginal success will mean the heroes are treated as guests but are not involved in the elders' discussions of what to do next. Higher levels of success ensure the heroes are invited to participate in the discussions. ;Sample arguments from the elders * The Tongs always lie. How do we know you're not lying when you say your against the Tongs? * You say you were fighting the Tongs, but it was a strange fight that resulted in your enemy escaping unscathed. * Why were you travelling with those good-for-nothing Lo-Fak? * We never trust the Dragon-people over the mountains: they never tell the truth. You say you are working for them? Once the elders have discussed the heroes, the villagers will decide what to do next. Opinion is divided in the village. Karakol speaks for the elders and most of the other villagers in doing little more than remaining vigilant and allowing the heroes to go on their way. Tokmak speaks for the hunters and the more hot-headed villagers in pursuing the Tong enforcers and eliminating the problem. A discussion follows between the two sides. ;Karakol (Starting APs = 45) :Reasoned Argument 10W, Cautious 5W, Wisdom of Age 2W. ;Tokmak (Starting APs = 25) :Boast 20, Hotheaded 2W, Appeal to Honour 3W. The starting APs reflect the relative size of the two factions. The heroes can enter the debate on whichever side they want to: more fun will be had with some heroes on both sides of the argument. Each hero will add 1/10 (round down) of their most appropriate ability to the APs of the side they join. Add 3 APs if the heroes won a Complete success against the elders just before, and subtract 1 if they won only a Minor success. {| align="center" {{Prettytable}} |Reduce your workload: get a player to roleplay each of Karakol and Tokmak through the debate, with the other players making additional comments for their heroes. |} It is likely that Karakol's faction will win, and the villagers will go to sleep. However, during the dead of night, Tokmak and a few warriors will sneak out of the village to hunt down the Tong enforcers. If the heroes particularly impressed Tokmak, he may creep into their hut and invite them with his group. The villagers will not realise anyone is missing until the next morning. One of the very young men will say that Tokmak tried to persuade him to go with Tokmak, but he was too frightened. The village elders, remembering the stories told of the fight at Shinuan, fear that the Tong enforcers will defeat Tokmak and his group, and ask the heroes to lead a party to reinforce or rescue them. === Scene 2: Ambush! === ''Summary: Hothead Dimali youths are ambushed by the Tong, but learn that they are vulnerable to misdirection.'' The heroes must organise the party they are to lead. The best warriors and hunters have gone with Tokmak, leaving the gatherers, the youngesters, and the elders behind. The heroes must persuade the villagers to follow them: '''Persuade the villagers to join the expedition''' (18): ''Persuade, Dimali Customs (-5), Enthusiastic (-8).'' :Complete success: 10 villagers follow the heroes. :Major success: 6 villagers. :Minor success: 4 villagers. :Marginal success, tie, marginal failure: 2 villagers. :Minor failure: No-one follows the heroes, but they receive supplies. :Major failure: No-one goes with the heroes. :Complete failure: Not only do no villagers accompany the heroes, the elders blame the heroes for the situation and require hostages or valuable artefacts to be left behind as surety. :Treat the villagers as Followers with abilities of Local Knowledge 18 and Hunter 15. The best trackers have gone with Tokmak, but nevertheless it is easy to follow their trail across the valley. It doesn't take too long to realise that they are heading to meet the trail that was seen leaving Shinuan. As they begin to climb out of the last patch of woodland on the far side of the valley, the heroes make out the sounds of a fight. If the heroes succeed at a Sneak ability test, they can observe what is going on. Just along the edge of the patch of woodland, near a small stream, Tokmak and this band are fighting the Tong enforcers last seen in Shinuan. The enforcers have the better of the situation. Four of the nine Dimali warriors are already out of the fight, and the enforcers are using their movement abilities to keep the Dimali off balance and to strike at the warriors protecting their fallen comrades. The initial position is Tokmak and allies: 5 people, Ranged Combat 16 (5W-9 for multiple attackers) 50AP; Enforcers: 12 people, Leaping Combat 19W, 130AP. Allow the heroes to discuss their plans and augment their abilities, but all the while describe the ongoing fight and the damage being suffered by Tokmak. When the heroes do join the fight, they should be able to turn the tide. As at Shinuan, once the enforcers realise the situation is going badly for them, they will change their objective to leave the scene, taking all their comrades with them. After calm returns, a number of the Dimali are Injured and most are Hurt. Everyone needs to return to Asquat for healing and recuperation. On the way back, Tokmak will describe what happened. As the Dimali were coming out of the woodlands, they were ambushed by the enforcers and never really had a chance to recover from that initial shock. "They fight like devils!" Tokmak will say. The only respite they had was when ... (* which of the Four Vices to use? *) === Scene === Preparing tactics for the defeat of the Tong enforcers === Scene (Pinch I) === Tracking down and defeating the Tong enforcers. === Midpoint === The PCs have found the Tong hideout, and attack to prevent them wiping out any more villages. However, they should capture a prisoner who will reveal the true story: that the Tong people are there to destroy the Huan To, and they think the PCs are the Huan To's allies. The focus now shifts to finding the Huan To's hideout, and magical preparations of how to defeat him. === Scene === Predicting where the Huan To will strike next. Use of various ritual conjunctions to do this. === Scene === Being where the predicted strike will occur, and nothing happening. Realisation that this is now the final run up to the big ritual. === Scene === Preparation of ritual object to defeat Huan To's warding and get into his inner sanctum. === Scene === Discovery of the location of Huan To's lair. === Scene (Pinch II) === Infiltration of the Huan To's base. === Scene (Plot Turn II) === Heroes are captured by the Huan To. Also include his exposition monologue, including using the heroes' entrails as omens to be read at the next dawn. If he discovers the ritual object needed to defeat him, he lets slip where it is kept. == Act III: Resolution == The Heroes are inside the Huan To's lair, a large village with a maze of streets and buildings, and a collection of tunnels and caves below. The ghoul army is nearing completion, with large numbers of slavering ghouls kept penned up in the warehouses. If he is not stopped, the ghouls will be released and will ravage the lands around, heading over the mountains towards the heart of Kralorela. The PCs can also discover the use of talismans to identify legitimate inhabitants of the Huan To's lair. They can steal some of these, and use them to infiltrate the lair, only to be discovered at some point. ;Setting The Huan To's village :This is a large fortified village, with many stone buildings in the centre, including some warehouses. There are also caves and tunnels in the rock below. Stairs lead up and down the insides and outsides of the houses. Many alleyways are covered, and it is often easier to go from place to place across the rooftops, which have defined alleyways across them. === Scene 1: Escape === The climactic scene will involve the Heroes running through the village, being attacked by and defeating ghouls, to break into the Huan To's inner sanctum. The PCs will start by having to escape from their cell and defeat the ghoul guards. Then can then run through the village, searching for the inner sanctum and the HT. ;Hints for running this extended chase :Some actions are just too useful to pass up. Typical things occurring in this scene are: * Chasing a couple of ghouls down a corridor, only to turn a corner to be confronted by a horde of ghouls in a large room. * Having to cross a narrow wall between two courtyards, both filled with hordes of slavering ghouls. * Coming across a fellow prisoner, only to discover that he is now a ghoul. === Scene 2 === Once in the inner sanctum, the Huan To will escape into the catacombs below the village. The SDMP is in the inner sanctum, but protected by a crystal dome. The PCs will need to keep the ghoul army out while they hunt him down. Run this as an extended contest: the HT's actions will be to debate the nature of good and evil with the PCs. If any are defeated, they will question their own moral code, and express interest in Sekeveran. Eventually, the HT will be tracked down (defeated) and will appear back in his inner sanctum. His body will then explode in a soundless blast of white light, leaving only his exoskeleton behind. The PCs can then destroy the SDMP. The HT's arguments will be like extreme libertarianism and communism (Trotskyite?): * Property is theft * Leaders exist solely to preserve their own position * Why should individuals give up their rights to the use of force? This will cause all the ghouls to immediately decay and die. It will also cause the inner sanctum and catacombs to rumble and start to collapse. The PCs must then escape the village before it all becomes ruined. === Epilogue (Optional) === A hsunchen woman gives birth to a baby boy. As she does so, there is a flash of white light, and a shout from the village's guards of a bandit attack. The baby screams, to reveal its forked tongue... == Characters == === Shia Shan Gua: The Huan To === He is an ambitious and headstrong young Tong member. He saw an opportunity to kill his mentor and steal his magical artefacts. Using these has turned him into a Huan To, though he has little idea of the finer points of Sekeveran philosophy. He is hunted by the others in his Tong gang. He is attempting to raise a ghoul army, using the Claw of the Straining Heart to stabilise them, to attack Kralorela. He knows this will be suicidal, but he has little idea of what else to do. ;Abilities ;Grimoires === The Shinuan Villagers (temporary PCs) === === The Claw of the Straining Heart: The Strange Device of Mystic Power === This is an artefact that allows the ghouls to remain stable, without the need to constantly feed and possess corpses. This allows for a much larger ghoul force than otherwise. === Inspiration === The main motivation for this plot came from a reading from the I Ching, which indicated hexagram 53, with line 3 moving. The movement from hexagram 53 to hexagram 20 provided the movement in the plot. ;Hexagram 53: Chien (Gradual Progression) :A man advances from a lesser position to one to which he is better suited. Suitable time is taken for this advancement. :Line 3: He is headstrong, and apt to make a wrong decision. He alienates himself from those close to him. ;Hexagram 20: Kuan (Contemplation) :A great and gentle ruler who inspires others with his example. His preparation for a pious act inspires onlookers to follow him on the path of virtue. :Line 3: He is uncertain of his position, and is unsure whether to advance or retreat. He must stop and reflect to find the right choice. The three-act structure is a standard for screenwriting, and its use for dramatist scenario construction is advocated in the Theatrix role-playing game. [[Category:Scenarios]] [[Category:HeroQuest]] [[Category:Glorantha]] f34bcaa08eab6333d226df157d48d62e700a948d 0 1515 1959 1908 2009-02-21T22:43:33Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * ''[[Ruby on Rails setup]]'' | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. [[Category:server setup]] 0e300380c9223a3fd08056a0c7d5551df96654f2 1964 1959 2009-02-23T10:42:20Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * ''[[Ruby on Rails setup]]'' | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. [[Category:Server setup]] d33c59d8e94558d6764aac9cb97118507394cd8a 1993 1964 2009-03-04T12:58:30Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 2340f077cead70fe1ec1d6bb73dd93a8ca22f8bb 0 1570 1961 2009-02-23T10:40:30Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[DNS server setup|DNS]] |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |} NTP (network time protocol) is a way of sending time across the internet. If a machine has an NTP client, it will keep its clock synchronised with various atomic clocks. The default <code>ntupdate</code> system synchronises the clock on every boot, but for long-running machines the clock can drift quite a bit between boots. A better option is to install the <code>ntp</code> daemon, which reset the clock every few minutes to prevent it getting too far adrift. * Install the daemon root@server:~# apt-get install ntp * Update <code>/etc/ntp.conf</code> to point to nearby servers server ntp.ubuntu.com server uk.pool.ntp.org :(find a nearby server from the [http://www.pool.ntp.org list of NTP server pools]). * Restart the <code>ntp</code> daemon so it knows about the new server root@server:~# /etc/init.d/ntp restart * If the clock is a few minutes off, it may confuse other server processes when <code>ntp</code> resets it the first time, so you may need to restart those servers. It may be easier to just reboot the machine. == See also == * This is pretty much taken from [https://help.ubuntu.com/8.04/serverguide/C/NTP.html the notes in the Ubuntu server guide]. * The [http://support.ntp.org/bin/view/Support/WebHome NTP documentation] has more information. [[Category:Server setup]] c86d6133670ef3120ce6ee57799d42229b99b23a 0 1544 1962 1866 2009-02-23T10:41:04Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld NS desktop.domain.tld ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/bind/run root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run * Modify the file <tt>/etc/default/bind9</tt> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * Modify <tt>/etc/default/syslogd</tt> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * Make AppArmor allow BIND access to the chroot jail. Modify <tt>/etc/apparmor.d/usr.sbin.named</tt> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/run/bind/run/named.pid w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, Sysklogd, and BIND: root@server:~# /etc/init.d/apparmor restart root@server:~# /etc/init.d/sysklogd restart root@server:~# /etc/init.d/bind9 restart * Check <tt>/var/log/syslog</tt> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 9bff607bb55baedc79d6493672109304460cb52a 0 1530 1963 1760 2009-02-23T10:41:19Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <tt>/etc/ssl/domain</tt>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create the <tt>/etc/ssl/openssl.cnf</tt> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME = . RANDFILE = $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids # To use this configuration file with the "-extfile" option of the # "openssl x509" utility, name here the section containing the # X.509v3 extensions to use: # extensions = # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca' and 'req'. # Add a simple OID like this: # testoid1=1.2.3.4 # Or use config file substitution like this: # testoid2=${testoid1}.5.6 #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. #unique_subject = no # Set to 'no' to allow creation of # several ctificates with same subject. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/crl.pem # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert # Comment out the following two lines for the "traditional" # (and highly broken) format. name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options # Extension copying option: use with caution. # copy_extensions = copy # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs # so this is commented out by default to leave a V1 CRL. # crlnumber must also be commented out to leave a V1 CRL. # crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering # A few difference way of specifying how similar the request should look # For type CA, the listed attributes must be the same, and the optional # and supplied fields are just that :-) policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy # At this point in time, you must list all acceptable 'object' # types. [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert # Passwords for private keys if not present they will be prompted for # input_password = secret # output_password = secret # This sets a mask for permitted string types. There are several options. # default: PrintableString, T61String, BMPString. # pkix : PrintableString, BMPString. # utf8only: only UTF8Strings. # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). # MASK:XXXX a literal mask value. # WARNING: current versions of Netscape crash on BMPStrings or UTF8Strings # so use this option with caution! string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name # we can do this but it is not needed normally :-) #1.organizationName = Second Organization Name (eg, company) #1.organizationName_default = World Wide Web Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) #organizationalUnitName_default = commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 # SET-ex3 = SET extension number 3 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName [ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] # Extensions for a typical CA # PKIX recommendation. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always # This is what PKIX recommends but some broken software chokes on critical # extensions. #basicConstraints = critical,CA:true # So we do this instead. basicConstraints = CA:true # Key usage: this is typical for a CA certificate. However since it will # prevent it being used as an test self-signed certificate it is best # left out by default. # keyUsage = cRLSign, keyCertSign # Some might want this also # nsCertType = sslCA, emailCA # Include email address in subject alt name: another PKIX recommendation # subjectAltName=email:copy # Copy issuer details # issuerAltName=issuer:copy # DER hex encoding of an extension: beware experts only! # obj=DER:02:03 # Where 'obj' is a standard or added object # You can even override a supported extension: # basicConstraints= critical, DER:30:03:01:01:FF [ crl_ext ] # CRL extensions. # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. # issuerAltName=issuer:copy authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] # These extensions should be added when creating a proxy certificate # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName # This really needs to be in place for it to be a proxy certificate. proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] 3d94ce5cfb92438c377170767153f0d71dd70a4f 0 1571 1965 2009-02-23T15:48:04Z Neil 2 Redirecting to [[Category:RPGs]] wikitext text/x-wiki #REDIRECT [[:Category:RPGs]] dece4bf4cf7e31f9a09c9562532edbd395001541 1971 1965 2009-02-23T15:51:59Z Neil 2 Redirecting to [[Category:Glorantha]] wikitext text/x-wiki #REDIRECT [[:Category:Glorantha]] 2aff5aefb1fef3323c2648c2526601375e449875 0 1572 1966 2009-02-23T15:48:36Z Neil 2 Redirecting to [[Category:HeroQuest]] wikitext text/x-wiki #REDIRECT [[:Category:HeroQuest]] f6e00841c316f60b7047c1f78ce4112904053797 0 1573 1967 2009-02-23T15:49:15Z Neil 2 Redirecting to [[Category:Name generation]] wikitext text/x-wiki #REDIRECT [[:Category:Name generation]] a19adf55e99e8314d6f77eac9f1c94d833c2d513 0 1574 1968 2009-02-23T15:49:55Z Neil 2 Redirecting to [[Category:Scenarios]] wikitext text/x-wiki #REDIRECT [[:Category:Scenarios]] 2c95aee5d1d4f29aac3318a4f33e4d9681972b7e 0 1575 1969 2009-02-23T15:50:24Z Neil 2 Redirecting to [[Category:Tertorae]] wikitext text/x-wiki #REDIRECT [[:Category:Tertorae]] 23e583644b564d2d4f65459154c092a75b207534 0 1576 1970 2009-02-23T15:50:47Z Neil 2 Redirecting to [[Category:Yellowtail]] wikitext text/x-wiki #REDIRECT [[:Category:Yellowtail]] b6093a0704214376b70b4fcec49d4ccd351d5719 0 1526 1975 1930 2009-03-02T22:59:11Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] c178856e4ed0181db09990b7addba9b3f997491b 1976 1975 2009-03-02T23:02:24Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 84e0748cd64d0eefdc3565c3b3535888ef42282c 0 1577 1977 2009-03-04T10:13:53Z Neil 2 New page: How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[My... wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 [[Category:Ruby]] [[Category:Rails]] ebc7644837528c926f563b799ea032e6e3eae250 1980 1977 2009-03-04T10:18:19Z Neil 2 /* Installing Rails with NetBeans 6.5 on Windows */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 == Installing InstantRails and getting it working with Netbeans == [[Category:Ruby]] [[Category:Rails]] d3ac3f122cd3c4398c2d600c42895de2e22b40f7 0 1578 1978 2009-03-04T10:13:57Z Neil 2 New page: Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from ... wikitext text/x-wiki Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! [[Category:Ruby]] [[Category:Rails]] cf847fdae71e09fdd4de7e785b021e3e31b59dfc 0 1579 1979 2009-03-04T10:17:23Z Neil 2 New page: Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|data... wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations [[Category:Ruby]] [[Category:Rails]] d016f72c6966dad6da3ebc231309fbf243644438 1986 1979 2009-03-04T10:34:54Z Neil 2 /* Subsequent deployments with Capistrano from a stable branch */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations [[Category:Ruby]] [[Category:Rails]] e82f3c2ea23a5bc0ddf3d944da27f1c5d87da1c0 1987 1986 2009-03-04T11:20:40Z Neil 2 /* Serving pages with Apache and Mongrel cluser */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all # Uncomment this directive is you want to see apache2's # default start page (in /apache2-default) when you go to / #RedirectMatch ^/$ /apache2-default/ </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed other the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog logs/myapp_deflate_log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations [[Category:Ruby]] [[Category:Rails]] 38f478b60fe33c158442f1bd2140b92b4dd07ba1 1988 1987 2009-03-04T11:23:18Z Neil 2 /* Serving pages with Apache and Mongrel cluser */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed other the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog logs/myapp_deflate_log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations [[Category:Ruby]] [[Category:Rails]] 8e3cadd4e2f19f7d572ac4835f3c1a30ea7995b4 1989 1988 2009-03-04T11:24:40Z Neil 2 /* Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed other the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog logs/myapp_deflate_log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations [[Category:Ruby]] [[Category:Rails]] 99561300f66804063e2450c792a5f4686b9b0788 1990 1989 2009-03-04T11:25:24Z Neil 2 /* Serving pages with Apache and Mongrel cluser */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed other the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog logs/myapp_deflate_log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Modifications to Capfile * Running Mongrel as a service == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations [[Category:Ruby]] [[Category:Rails]] ea8096e3bf70815dfed5a216e16036de9199959d 1991 1990 2009-03-04T11:29:05Z Neil 2 wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed other the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog logs/myapp_deflate_log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Modifications to Capfile * Running Mongrel as a service == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 6a8bf4a63e80fd1f7ee15f6c086ff611c95a33c0 1992 1991 2009-03-04T12:54:04Z Neil 2 /* Serving pages with Apache and Mongrel cluser */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed other the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog logs/myapp_deflate_log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === * Modifications to Capfile === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Comment out this line near the top of the script: # USER=mongrel * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] bfb8070645279695ff03aa5afd6dc6e598f603b4 1994 1992 2009-03-04T12:59:04Z Neil 2 /* Make Apache a proxy for Mongrel */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog logs/myapp_deflate_log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === * Modifications to Capfile === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Comment out this line near the top of the script: # USER=mongrel * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 2781bea110616aa68526266ce31c91c66a2f1371 1997 1994 2009-03-04T13:27:43Z Neil 2 /* Make Apache a proxy for Mongrel */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === * Modifications to Capfile === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Comment out this line near the top of the script: # USER=mongrel * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 9fe8e232da5c21aa111cbeef18fab32e3bf7143c 0 1552 1982 1828 2009-03-04T10:21:59Z Neil 2 wikitext text/x-wiki I have written a couple of small programs: * A Tcl/Tk script that shows [http://scripts.njae.me.uk/downloads/number numbers and words] (for teaching children). * A Tcl/Tk script that poses [http://scripts.njae.me.uk/downloads/sums simple sums] (for teaching children). If you want to change the difficulties of the levels, you'll need to edit the parameters at the top of the script. * A Delphi (Windows 95) program that [http://scripts.njae.me.uk/downloads/name-generation/namegen.zip generates random words] by combining syllables. See the [[:Category:Name generation|name generation page]] for more details. * A Tcl script that [http://scripts.njae.me.uk/downloads/name-generation/ngramwords generates random words] by building an ''n''-gram model of a language. See the [[ngramwords documentation| documentation]] and the [[:Category:Name generation|name generation page]] for more details. These programs are released under the [http://scripts.njae.me.uk/downloads/gpl.txt GNU Public Licence (GPL)], meaning that they are free and open-source software. [[Category:Programming]] 7c8721caf27a3dea0b0fb5d90602d573a9706ad7 14 1580 1983 2009-03-04T10:22:48Z Neil 2 New page: [[Category:Programming]] wikitext text/x-wiki [[Category:Programming]] aed2406a3a618eb93893d3cb5749d13c4a2192ac 14 1581 1984 2009-03-04T10:22:50Z Neil 2 New page: [[Category:Programming]] wikitext text/x-wiki [[Category:Programming]] aed2406a3a618eb93893d3cb5749d13c4a2192ac 14 1582 1985 2009-03-04T10:24:26Z Neil 2 New page: Pages related to programming. See also [[Server setup]] for setting up server machines. wikitext text/x-wiki Pages related to programming. See also [[Server setup]] for setting up server machines. 0714e9ca554b17a067689f4e0a56c242a04b8752 0 1533 1996 1836 2009-03-04T13:23:25Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <tt>/etc/apache2/sites-available</tt>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <tt>ServerName</tt> and <tt>DocumentRoot</tt> settings to reflect the site's name and location of files. Also note the non-default location of the <tt>cgi-bin</tt> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <tt>root</tt>, and permissions of 755. Also note that the contents of the <tt>/usr/share/doc/</tt> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 restart == Secure HTTP == I use the Secure HTTP server for my webmail. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <tt>/etc/apache2/ports.conf</tt> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <tt><nowiki>www.domain.tld</nowiki></tt> == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here) [[Category:Server setup]] 73f42279fafc57e5114ec0a5baa78585e0d7ac28 1998 1996 2009-03-04T13:28:07Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <tt>/etc/apache2/sites-available</tt>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <tt>ServerName</tt> and <tt>DocumentRoot</tt> settings to reflect the site's name and location of files. Also note the non-default location of the <tt>cgi-bin</tt> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <tt>root</tt>, and permissions of 755. Also note that the contents of the <tt>/usr/share/doc/</tt> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 restart == Secure HTTP == I use the Secure HTTP server for my webmail. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <tt>/etc/apache2/ports.conf</tt> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <tt><nowiki>www.domain.tld</nowiki></tt> == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 5f5312878f0eb58a975f70075b58af9bcc895fba 0 1577 1999 1980 2009-03-04T17:16:45Z Neil 2 wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 == Installing InstantRails and getting it working with Netbeans == == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. [[Category:Ruby]] [[Category:Rails]] 9909e22bd8f248268943e24616791b145fff65ee 2033 1999 2009-03-09T21:49:19Z Neil 2 wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 866d761f975ebfcebbec78f985344ad9dd0020eb 2046 2033 2009-03-11T19:56:42Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] a7ab8e7133f6215f839ae341d8cd7b30f5042859 2047 2046 2009-03-11T19:59:48Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 912bfc03c297432dc45df0fa4f4c34b39d222a1f 0 1579 2000 1997 2009-03-04T17:30:58Z Neil 2 /* Make Mongrel run as a service */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi Options +FollowSymLinks +ExecCGI RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> </Directory> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === * Modifications to Capfile === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 7101c16c807195fa9cb2d2248903123d61771b03 2001 2000 2009-03-04T21:37:44Z Neil 2 /* Bare bones deployment: Apache2 and mod_ruby */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === * Modifications to Capfile === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] cd772024101d0e1ebeb515d6b4ef599b0b924ca3 2012 2001 2009-03-05T08:09:12Z Neil 2 /* Update the project's capfile */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. However, as I'm running Mongrel as a service, and I'm intending to use Passenger for real deployments, I think I'll ignore this for the moment. === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 83c0e9bbb57627371bf6a73cf6ed273dbdaac5d5 2013 2012 2009-03-05T08:13:14Z Neil 2 /* Make Mongrel run as a service */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. However, as I'm running Mongrel as a service, and I'm intending to use Passenger for real deployments, I think I'll ignore this for the moment. === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 54cb30cd367622769443588b5d6d1f6d713e4385 2027 2013 2009-03-09T13:50:30Z Neil 2 /* Update the project's capfile */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Define the user for running and stopping server sub-proceses (Mongrel, etc.) set :runner, :name_of_some_user # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 44d312c8c96ce4b5f9815e40bb3d4ec8906a577b 2028 2027 2009-03-09T14:14:22Z Neil 2 /* Update the project's capfile */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 892947277b82f4470d43fa78ea7ee320d13c3ecf 2048 2028 2009-03-11T20:00:27Z Neil 2 /* Subsequent deployments with Capistrano from a stable branch */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Serving pages with Apache and Mongrel cluser == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 + Phusion Passenger == == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] b35a60007f5f4e7eb5ab79a9662de5481811cd43 0 1583 2002 2009-03-05T00:43:55Z Neil 2 New page: The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the ba... wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === First, create a migration to add the new field user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date The migration file itself shouldn't need any modification === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all<span style="background-color:#f9bbbb;">, :conditions => "date_available <= now()"</span>) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == == Order the product catalogue and cart == == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] ec6a3dbbdb46f79333e0ebc544208e89c5824383 2003 2002 2009-03-05T00:45:29Z Neil 2 /* Order the product catalogue and cart */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === First, create a migration to add the new field user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date The migration file itself shouldn't need any modification === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all<span style="background-color:#f9bbbb;">, :conditions => "date_available <= now()"</span>) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == == Present items in product catalogue and cart in alphabetical order == == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 5a848141fae4e5b31b38bc71b2246252a65a1404 2004 2003 2009-03-05T00:46:24Z Neil 2 /* Present items in product catalogue and cart in alphabetical order */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === First, create a migration to add the new field user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date The migration file itself shouldn't need any modification === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all<span style="background-color:#f9bbbb;">, :conditions => "date_available <= now()"</span>) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == == Sort cart items alphabetically == == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] b34c78e884065732eab288796921a4e514b4ee23 2005 2004 2009-03-05T00:48:05Z Neil 2 /* Update the Product model to include the date constraint */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === First, create a migration to add the new field user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date The migration file itself shouldn't need any modification === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == == Sort cart items alphabetically == == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] b5d1a75c5301debd6f214cdf7c14afb22b201216 2006 2005 2009-03-05T00:50:24Z Neil 2 /* Create a migration to add the 'date available' field */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == == Sort cart items alphabetically == == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] af69bc5be8a095a84b65d976b0dcf3b04c00f6b4 2007 2006 2009-03-05T00:54:33Z Neil 2 /* Create a migration to add the 'date available' field */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == == Sort cart items alphabetically == == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 9f504bb1807a2a6833f447d108a0f9b4c51a0913 2008 2007 2009-03-05T01:09:46Z Neil 2 /* Add a date products are available until */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] be82d1d83ce303c129c58103db5adedecff3ffaf 2009 2008 2009-03-05T01:15:10Z Neil 2 /* Sort cart items alphabetically */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 9a1689e9329de1d6bd7e2209c1639253c078307b 2010 2009 2009-03-05T01:17:47Z Neil 2 /* Improve the decoupling between cart items and order items */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) [[Category:Ruby]] [[Category:Rails]] c97e2e408e5a883d93cdf4d83c2cb9ad3ab7c054 2011 2010 2009-03-05T08:06:58Z Neil 2 /* Improve the decoupling between cart items and order items */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] c739f74473b1770b29af3583add67338095b7cd4 2014 2011 2009-03-05T09:37:41Z Neil 2 /* Create a migration to add the 'date available' field */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 4994952dd1a44353692c421ed7caebfb010a68ea 2017 2014 2009-03-06T17:35:39Z Neil 2 /* Create a migration to add the 'date available' field */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span>Product.reset_column_information end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] e01f4622e3555a25d444d01b39dec7d480031673 2018 2017 2009-03-06T17:35:58Z Neil 2 /* Create a migration to add the 'date available' field */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span>Product.reset_column_information end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] f8a861be396fcd75c300135d581ac3c2bb71e98b 2019 2018 2009-03-06T17:38:28Z Neil 2 /* Update the Product view forms */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span>Product.reset_column_information end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><b>Date available:</b></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><%= f.label :date_available %><br /></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><%= f.label :date_available %><br /></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] c28a6ab636368e4c97d4967009a2205d377510b3 2021 2019 2009-03-06T17:57:18Z Neil 2 /* Create a migration to add the 'date available' field */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date {{Code highlight}}Product.reset_column_information {{Code highlight}}end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><b>Date available:</b></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><%= f.label :date_available %><br /></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><p></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><%= f.label :date_available %><br /></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <span style="float:left; width:0px;">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span><nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 80042d4a7f9476650c1f6cf4f6a060e57b6334f7 2022 2021 2009-03-06T17:59:56Z Neil 2 /* Update the Product view forms */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date {{Code highlight}}Product.reset_column_information {{Code highlight}}end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available:</b></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available.past? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available since <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable <span style="background-color:#f9bbbb;"><nowiki>#product-list .list-description .unavailable {</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>color: #f00;</nowiki></span> <span style="background-color:#f9bbbb;"><nowiki>}</nowiki></span> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, <span style="background-color:#f9bbbb;">:conditions => "date_available <= now()",</span> :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date <span style="background-color:#f9bbbb;">Product.reset_column_information</span> end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><b>Date available until:</b></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%=h @product.date_available_until %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> <span style="background-color:#f9bbbb;"><nowiki><p></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.label :date_available_until %><br /></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki></p></nowiki></span> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> <span style="background-color:#f9bbbb;"><nowiki><% if not product.date_available_until.nil? %> </nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% if product.date_available_until.future? %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% else %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <span style="background-color:#f9bbbb;"><nowiki><% end %></nowiki></span> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() <span style="background-color:#f9bbbb;">and (date_available_until is null or date_available_until >= now())</span>", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items<span style="background-color:#f9bbbb;">.sort_by {|item| item.product.title}</span>) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 8d71d5834a38984cc3ca865f6f41d2217b258b9e 2023 2022 2009-03-06T18:29:30Z Neil 2 wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date {{Code highlight}}Product.reset_column_information {{Code highlight}}end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available:</b></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> {{Code highlight}}<nowiki><% if product.date_available.past? %></nowiki> {{Code highlight}}<nowiki><dd>Available since <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable {{Code highlight}}<nowiki>#product-list .list-description .unavailable {</nowiki> {{Code highlight}}<nowiki>color: #f00;</nowiki> {{Code highlight}}<nowiki>}</nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, {{Code highlight}}:conditions => "date_available <= now()", :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date {{Code highlight}}Product.reset_column_information end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available until:</b></nowiki> {{Code highlight}}<nowiki><%=h @product.date_available_until %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% if not product.date_available_until.nil? %> </nowiki> {{Code highlight}}<nowiki><% if product.date_available_until.future? %></nowiki> {{Code highlight}}<nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() {{Code highlight}}and (date_available_until is null or date_available_until >= now())", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items{{Code highlight}}.sort_by {|item| item.product.title}) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == AJAXify the checkout form == If you've been following along the book, doing all the additional extras, you'll have a fully AJAXified cart. Unfortunately, that means that it won't work properly with the checkout code as presented in the book. To make it work, you need to make the following modifications. * Add the 'Checkout' button to <code>_cart.html.erb</code> <nowiki><div class="cart-title">Your Cart</div></nowiki> <nowiki><table></nowiki> <nowiki><%= render(:partial => 'cart_item', :collection => cart.items.sort_by {|item| item.product.title}) %></nowiki> <nowiki><tr class="total-line"></nowiki> <nowiki><td colspan="2">Total</td></nowiki> <nowiki><td class="total-cell"><%= number_to_currency(cart.total_price, :unit => "£") %></td></nowiki> <nowiki></tr></nowiki> <nowiki></table></nowiki> {{Code highlight}}<nowiki><% form_remote_tag :url => {:action => 'checkout'} do %></nowiki> {{Code highlight}}<nowiki><%= submit_tag "Checkout" %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki><% form_remote_tag :url => {:action => 'empty_cart'} do %></nowiki> <nowiki><%= submit_tag "Empty cart" %></nowiki> <nowiki><% end %></nowiki> == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] d05607cf2e8a3a7e01186a0281f9c74488dbc533 2024 2023 2009-03-06T19:42:07Z Neil 2 /* AJAXify the checkout form */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date {{Code highlight}}Product.reset_column_information {{Code highlight}}end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available:</b></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> {{Code highlight}}<nowiki><% if product.date_available.past? %></nowiki> {{Code highlight}}<nowiki><dd>Available since <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable {{Code highlight}}<nowiki>#product-list .list-description .unavailable {</nowiki> {{Code highlight}}<nowiki>color: #f00;</nowiki> {{Code highlight}}<nowiki>}</nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, {{Code highlight}}:conditions => "date_available <= now()", :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date {{Code highlight}}Product.reset_column_information end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available until:</b></nowiki> {{Code highlight}}<nowiki><%=h @product.date_available_until %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% if not product.date_available_until.nil? %> </nowiki> {{Code highlight}}<nowiki><% if product.date_available_until.future? %></nowiki> {{Code highlight}}<nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() {{Code highlight}}and (date_available_until is null or date_available_until >= now())", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items{{Code highlight}}.sort_by {|item| item.product.title}) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == AJAXify the checkout form == If you've been following along the book, doing all the additional extras, you'll have a fully AJAXified cart. Unfortunately, that means that it won't work properly with the checkout code as presented in the book. To make it work, you need to make the following modifications. * Add the 'Checkout' button to <code>_cart.html.erb</code> <nowiki><div class="cart-title">Your Cart</div></nowiki> <nowiki><table></nowiki> <nowiki><%= render(:partial => 'cart_item', :collection => cart.items.sort_by {|item| item.product.title}) %></nowiki> <nowiki><tr class="total-line"></nowiki> <nowiki><td colspan="2">Total</td></nowiki> <nowiki><td class="total-cell"><%= number_to_currency(cart.total_price, :unit => "£") %></td></nowiki> <nowiki></tr></nowiki> <nowiki></table></nowiki> {{Code highlight}}<nowiki><% form_remote_tag :url => {:action => 'checkout'} do %></nowiki> {{Code highlight}}<nowiki><%= submit_tag "Checkout" %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki><% form_remote_tag :url => {:action => 'empty_cart'} do %></nowiki> <nowiki><%= submit_tag "Empty cart" %></nowiki> <nowiki><% end %></nowiki> * Modify the <code>checkout</code> action in the <code>store_controller</code> to respond properly to an xhr request. def checkout @cart = find_cart if @cart.items.empty? redirect_to_index("Your cart is empty" ) else @order = Order.new {{Code highlight}}respond_to do |format| {{Code highlight}}format.js if request.xhr? {{Code highlight}}format.html {{Code highlight}}end end end The desired action is for the 'Checkout' button to replace the catalogue display with the checkout form. That means we need a way of identifying the catalogue display so it can be replaced and something to replace it with. We also need to ensure that the checkout form display works both with and without JavaScript. * First, identify the main panel where the catalogue display is. Modify <code>app/views/store/index.html.erb</code> to wrap the display in a <code><nowiki><div></nowiki></code>: {{Code highlight}}<nowiki><div id="main_panel">/nowiki> <nowiki><h1>Neil's Whimsical Store Catalogue</h1>/nowiki> <nowiki><% for product in @products -%></nowiki> <nowiki><div class="entry"></nowiki> <nowiki><% form_remote_tag :url => {:action => 'add_to_cart', :id => product} do %></nowiki> <nowiki><%= image_submit_tag(product.image_url) %></nowiki> <nowiki><% end %></nowiki> <nowiki><h3><%= h(product.title) %></h3></nowiki> <nowiki><%= product.description %></nowiki> <nowiki><div class="price-line"></nowiki> <nowiki><span class="price"><%= number_to_currency product.price, :unit => "£" %></span></nowiki> <nowiki><% form_remote_tag :url => {:action => 'add_to_cart', :id => product} do %></nowiki> <nowiki><%= submit_tag "Add to cart" %></nowiki> <nowiki><% end %></nowiki> <nowiki></div></nowiki> <nowiki></div></nowiki> <nowiki><% end %></nowiki> {{Code highlight}}<nowiki></div></nowiki> * For the checkout action to render properly when called as an AJAX operation, it needs a partial to render. This partial should contain the whole checkout form, so rename <code>app/views/checkout.html.erb</code> as <code>app/views/_checkout.html.erb</code> * The <code>app/views/checkout.html.erb</code> view needs to render this partial. This means that non-AJAX calls will render it correcly <%= render(:partial => "checkout", :object => @order) %> * Create <code>app/views/checkout.js.rsj</code> to handle displaying the checkout form when called by AJAX page.replace_html("main_panel", :partial => "checkout", :object => @order) :This replaces the contents of the main panel in the standard layout with the partial. == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 5c8e462d503d024d5f5c9be1ecbde6c9302a16b4 2025 2024 2009-03-07T08:00:16Z Neil 2 /* AJAXify the checkout form */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date {{Code highlight}}Product.reset_column_information {{Code highlight}}end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available:</b></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> {{Code highlight}}<nowiki><% if product.date_available.past? %></nowiki> {{Code highlight}}<nowiki><dd>Available since <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable {{Code highlight}}<nowiki>#product-list .list-description .unavailable {</nowiki> {{Code highlight}}<nowiki>color: #f00;</nowiki> {{Code highlight}}<nowiki>}</nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, {{Code highlight}}:conditions => "date_available <= now()", :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date {{Code highlight}}Product.reset_column_information end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available until:</b></nowiki> {{Code highlight}}<nowiki><%=h @product.date_available_until %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% if not product.date_available_until.nil? %> </nowiki> {{Code highlight}}<nowiki><% if product.date_available_until.future? %></nowiki> {{Code highlight}}<nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() {{Code highlight}}and (date_available_until is null or date_available_until >= now())", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items{{Code highlight}}.sort_by {|item| item.product.title}) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == AJAXify the checkout form == If you've been following along the book, doing all the additional extras, you'll have a fully AJAXified cart. Unfortunately, that means that it won't work properly with the checkout code as presented in the book. To make it work, you need to make the following modifications. * Add the 'Checkout' button to <code>_cart.html.erb</code> <nowiki><div class="cart-title">Your Cart</div></nowiki> <nowiki><table></nowiki> <nowiki><%= render(:partial => 'cart_item', :collection => cart.items.sort_by {|item| item.product.title}) %></nowiki> <nowiki><tr class="total-line"></nowiki> <nowiki><td colspan="2">Total</td></nowiki> <nowiki><td class="total-cell"><%= number_to_currency(cart.total_price, :unit => "£") %></td></nowiki> <nowiki></tr></nowiki> <nowiki></table></nowiki> {{Code highlight}}<nowiki><% form_remote_tag :url => {:action => 'checkout'} do %></nowiki> {{Code highlight}}<nowiki><%= submit_tag "Checkout" %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki><% form_remote_tag :url => {:action => 'empty_cart'} do %></nowiki> <nowiki><%= submit_tag "Empty cart" %></nowiki> <nowiki><% end %></nowiki> * Modify the <code>checkout</code> action in the <code>store_controller</code> to respond properly to an xhr request. def checkout @cart = find_cart if @cart.items.empty? redirect_to_index("Your cart is empty" ) else @order = Order.new {{Code highlight}}respond_to do |format| {{Code highlight}}format.js if request.xhr? {{Code highlight}}format.html {{Code highlight}}end end end The desired action is for the 'Checkout' button to replace the catalogue display with the checkout form. That means we need a way of identifying the catalogue display so it can be replaced and something to replace it with. We also need to ensure that the checkout form display works both with and without JavaScript. * First, identify the main panel where the catalogue display is. Modify <code>app/views/layouts/store.html.erb</code> to wrap the main catalogue display in a <code><nowiki><div></nowiki></code>: ... <nowiki><div id="main"></nowiki> <nowiki><% if flash[:notice] -%></nowiki> <nowiki><div id="notice"><%= flash[:notice] %></div></nowiki> <nowiki><% end -%></nowiki> {{Code highlight}}<nowiki><div id="main_panel"></nowiki> <nowiki><%= yield :layout %></nowiki> {{Code highlight}}<nowiki></div></nowiki> <nowiki></div></nowiki> ... * For the checkout action to render properly when called as an AJAX operation, it needs a partial to render. This partial should contain the whole checkout form, so rename <code>app/views/checkout.html.erb</code> as <code>app/views/_checkout.html.erb</code> * The <code>app/views/checkout.html.erb</code> view needs to render this partial. This means that non-AJAX calls will render it correctly. The whole view should be this one line: <%= render(:partial => "checkout", :object => @order) %> * Create <code>app/views/checkout.js.rsj</code> to handle displaying the checkout form when called by AJAX page.replace_html("main_panel", :partial => "checkout", :object => @order) :This replaces the contents of the main panel in the standard layout with the partial. == Improve the decoupling between cart items and order items == '''Not done yet''' (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] f9b9e2d593b30926f97ae8e88b8370b631bd60c7 2026 2025 2009-03-09T11:17:04Z Neil 2 /* Improve the decoupling between cart items and order items */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date {{Code highlight}}Product.reset_column_information {{Code highlight}}end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available:</b></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> {{Code highlight}}<nowiki><% if product.date_available.past? %></nowiki> {{Code highlight}}<nowiki><dd>Available since <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable {{Code highlight}}<nowiki>#product-list .list-description .unavailable {</nowiki> {{Code highlight}}<nowiki>color: #f00;</nowiki> {{Code highlight}}<nowiki>}</nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, {{Code highlight}}:conditions => "date_available <= now()", :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date {{Code highlight}}Product.reset_column_information end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available until:</b></nowiki> {{Code highlight}}<nowiki><%=h @product.date_available_until %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% if not product.date_available_until.nil? %> </nowiki> {{Code highlight}}<nowiki><% if product.date_available_until.future? %></nowiki> {{Code highlight}}<nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() {{Code highlight}}and (date_available_until is null or date_available_until >= now())", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items{{Code highlight}}.sort_by {|item| item.product.title}) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == AJAXify the checkout form == If you've been following along the book, doing all the additional extras, you'll have a fully AJAXified cart. Unfortunately, that means that it won't work properly with the checkout code as presented in the book. To make it work, you need to make the following modifications. * Add the 'Checkout' button to <code>_cart.html.erb</code> <nowiki><div class="cart-title">Your Cart</div></nowiki> <nowiki><table></nowiki> <nowiki><%= render(:partial => 'cart_item', :collection => cart.items.sort_by {|item| item.product.title}) %></nowiki> <nowiki><tr class="total-line"></nowiki> <nowiki><td colspan="2">Total</td></nowiki> <nowiki><td class="total-cell"><%= number_to_currency(cart.total_price, :unit => "£") %></td></nowiki> <nowiki></tr></nowiki> <nowiki></table></nowiki> {{Code highlight}}<nowiki><% form_remote_tag :url => {:action => 'checkout'} do %></nowiki> {{Code highlight}}<nowiki><%= submit_tag "Checkout" %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki><% form_remote_tag :url => {:action => 'empty_cart'} do %></nowiki> <nowiki><%= submit_tag "Empty cart" %></nowiki> <nowiki><% end %></nowiki> * Modify the <code>checkout</code> action in the <code>store_controller</code> to respond properly to an xhr request. def checkout @cart = find_cart if @cart.items.empty? redirect_to_index("Your cart is empty" ) else @order = Order.new {{Code highlight}}respond_to do |format| {{Code highlight}}format.js if request.xhr? {{Code highlight}}format.html {{Code highlight}}end end end The desired action is for the 'Checkout' button to replace the catalogue display with the checkout form. That means we need a way of identifying the catalogue display so it can be replaced and something to replace it with. We also need to ensure that the checkout form display works both with and without JavaScript. * First, identify the main panel where the catalogue display is. Modify <code>app/views/layouts/store.html.erb</code> to wrap the main catalogue display in a <code><nowiki><div></nowiki></code>: ... <nowiki><div id="main"></nowiki> <nowiki><% if flash[:notice] -%></nowiki> <nowiki><div id="notice"><%= flash[:notice] %></div></nowiki> <nowiki><% end -%></nowiki> {{Code highlight}}<nowiki><div id="main_panel"></nowiki> <nowiki><%= yield :layout %></nowiki> {{Code highlight}}<nowiki></div></nowiki> <nowiki></div></nowiki> ... * For the checkout action to render properly when called as an AJAX operation, it needs a partial to render. This partial should contain the whole checkout form, so rename <code>app/views/checkout.html.erb</code> as <code>app/views/_checkout.html.erb</code> * The <code>app/views/checkout.html.erb</code> view needs to render this partial. This means that non-AJAX calls will render it correctly. The whole view should be this one line: <%= render(:partial => "checkout", :object => @order) %> * Create <code>app/views/checkout.js.rsj</code> to handle displaying the checkout form when called by AJAX page.replace_html("main_panel", :partial => "checkout", :object => @order) :This replaces the contents of the main panel in the standard layout with the partial. == Improve the decoupling between cart items and order items == (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) As written in the book, the store controller's save order action creates a new order and passes it the cart. The order model then creates a new line item for each item in the cart (requiring that the order model know about the internal structure of the cart) and the creation of the line item is the responsibility of the line item model, which has to know about the internal structure of the cart item. This is rather a lot of coupling. It's better for the order, line item, cart, and cart item models each to know very little about each other. The only place in the application that knows about both carts and orders is the store controller, so this is were the transfer of information from one to the other should take place. * Remove the <code>add_line_items_from_cart(cart)</code> procedure from <code>app/models/line_item.rb</code> * Remove the <code>self.from_cart_item(cart_item)</code> procedure from <code>app/models/line_item.rb</code> * Modify the <code>save_order</code> procedure in <code>app/controllers/store.rb</code>. Remove the call to <code>@order.add_line_items_from_cart(@cart)</code>. Add the order creation code shown below. def save_order @cart = find_cart @order = Order.new(params[:order]) {{Code highlight}}@cart.items.each do |item| {{Code highlight}}li = LineItem.new {{Code highlight}}li.product = item.product {{Code highlight}}li.quantity = item.quantity {{Code highlight}}li.total_price = item.price {{Code highlight}}@order.line_items << li {{Code highlight}}end if @order.save session[:cart] = nil redirect_to_index("Thank you for your order") else render :action => 'checkout' end end == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 62c0fbea510af1b7396a138d7d2dd349e1f73e0c 0 1526 2015 1976 2009-03-06T17:09:10Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 85c0a229dfdc2ea9f33f76dcde43be242279b7e8 2036 2015 2009-03-11T09:42:03Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| None |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 4b48e2726b0ce7ce691d604edd988229ee4faf0d 2041 2036 2009-03-11T14:26:38Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 0c63fd7775dfb2424829e8e142a73d326cd242d0 6 1584 2016 2009-03-06T17:28:16Z Neil 2 wikitext text/x-wiki da39a3ee5e6b4b0d3255bfef95601890afd80709 10 1585 2020 2009-03-06T17:56:29Z Neil 2 New page: <includeonly><span style="float:left; width:0px; position:relative; left:-0.7em">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span></includeonly><noinclude> '''Usage:''' <nowiki>... wikitext text/x-wiki <includeonly><span style="float:left; width:0px; position:relative; left:-0.7em">[[Image:Line-highlighter.png||8px|height=1em|link=]]</span></includeonly><noinclude> '''Usage:''' <nowiki>{{Code hightlight}}Line of code to highlight</nowiki> This template adds a small coloured triangle to the left of a line of code, to highlight that line in a block of code. </noinclude> 312e65b694be9991e08a992ac53822618b4db839 0 1578 2029 1978 2009-03-09T14:30:37Z Neil 2 /* Production database */ wikitext text/x-wiki Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should have done this already for you. Add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! [[Category:Ruby]] [[Category:Rails]] 16b7478328341cea70b995dc922c966d8f4aec57 0 1543 2030 1840 2009-03-09T14:35:17Z Neil 2 /* Iptables.rules */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> === Iptables.rules === This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END === Iptables script === #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] b50c8d11303b9c8ad42c0402b2ddc537dbcd6b21 0 1545 2031 1864 2009-03-09T14:38:56Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server == Passwords == The first thing is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <tt>hostname</tt> is the name of this host, e.g. <tt>server.domain.tld</tt>. * Add passwords to the <tt>root</tt> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); == Remote access == If you want to access MySQL databases from a remote machine, you'll need to have MySQL accept connections across the network. Edit <code>/etc/mysql/my.cnf</code> and change the <code>bind-address</code> entry to be the IP of the machine: bind-address = 192.168.2.252 The restart MySQL: root@server:~# /etc/init.d/mysql restart == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] b037cafd63ad0b20e7775bffc9da04b587279f52 2032 2031 2009-03-09T14:45:20Z Neil 2 /* Remote access */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server == Passwords == The first thing is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <tt>hostname</tt> is the name of this host, e.g. <tt>server.domain.tld</tt>. * Add passwords to the <tt>root</tt> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); == Remote access == If you want to access MySQL databases from a remote machine, you'll need to have MySQL accept connections across the network. Edit <code>/etc/mysql/my.cnf</code> and change the <code>bind-address</code> entry to be the IP of the machine: bind-address = 192.168.2.252 The restart MySQL: root@server:~# /etc/init.d/mysql restart Note that the [[Firewall setup|firewall]] is configured to only allow connections to MySQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] e2a935eec6b6f489b0855e33920fbbe8374e3097 0 1586 2034 2009-03-09T22:09:37Z Neil 2 New page: == ''Getting Things Done'' tracker == * Needs tags == Report writing tracker == * Complex data structure * Dashboard view * Control over who gets to see and edit what * Emailed reminders ... wikitext text/x-wiki == ''Getting Things Done'' tracker == * Needs tags == Report writing tracker == * Complex data structure * Dashboard view * Control over who gets to see and edit what * Emailed reminders * File upload * PDF formatting of reports == Child progress monitor == * Children, milestones * Historical dimension of attainment: show progress over time == Chldminder invoice system == * Email invoices * Payment integration * Selective viewing * Task requests * Calendar-based view == Forum system == * Forums * User settings * Private messaging * Selective editing of posts * BBCode parsing * Forum subscription == Moodle forum RSS generator == * Receive emails * Generate RSS == Course support system == * Who's doing which project * Who's generated feedback * Marks, comments, etc. [[Category:Ruby]] [[Category:Rails]] c75bf3a67e2d03e74b4953aea984bd6689aafea0 2035 2034 2009-03-09T22:23:48Z Neil 2 wikitext text/x-wiki == ''Getting Things Done'' tracker == * Needs tags == Report writing tracker == * Complex data structure * Dashboard view * Control over who gets to see and edit what * Emailed reminders * File upload * PDF formatting of reports == Child progress monitor == * Children, milestones * Historical dimension of attainment: show progress over time == Chldminder invoice system == * Email invoices * Payment integration * Selective viewing * Task requests * Calendar-based view == Forum system == * Forums * User settings * Private messaging * Selective editing of posts * BBCode parsing * Forum subscription == Moodle forum RSS generator == * Receive emails * Generate RSS == Course support system == * Who's doing which project * Who's generated feedback * Marks, comments, etc. == Agile development user story tracker == * Assign priorities * Show how completed * Show tests generated [[Category:Ruby]] [[Category:Rails]] 5de8b97865b15e16e897a3bf8c12dec4cb1fe089 2037 2035 2009-03-11T11:43:08Z Neil 2 wikitext text/x-wiki == ''Getting Things Done'' tracker == * Needs tags == Report writing tracker == * Complex data structure * Dashboard view * Control over who gets to see and edit what * Emailed reminders * File upload * PDF formatting of reports == Child progress monitor == * Children, milestones * Historical dimension of attainment: show progress over time == Chldminder invoice system == * Email invoices * Payment integration * Selective viewing * Task requests * Calendar-based view == Forum system == * Forums * User settings * Private messaging * Selective editing of posts * BBCode parsing * Forum subscription == Moodle forum RSS generator == * Receive emails * Generate RSS == Course support system == * Who's doing which project * Who's generated feedback * Marks, comments, etc. == Agile development user story tracker == * Assign priorities * Show how completed * Show tests generated == Games day organiser == * Register attendees * Record games on offer, with times * Allocate players to games * Blog of updates [[Category:Ruby]] [[Category:Rails]] 7a86b151edf23d3cdb4dc1872bdc8849e8f44b93 0 1587 2038 2009-03-11T14:03:30Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} == GnuPG == == x509 certificates == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM == * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax === Testing === == See also == * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] 16c33aad023fb0c2fe39cfa3ee68cb765b025aac 2039 2038 2009-03-11T14:03:45Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} == GnuPG == == x509 certificates == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax === Testing === == See also == * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] dcf8999579c5f7f624c3110a8d99b1dad3ef2426 2040 2039 2009-03-11T14:25:55Z Neil 2 /* x509 certificates */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} == GnuPG == == x509 certificates == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * Prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == See also == * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] ae1da444320c5265ae3f9d8c42c18fe6f92836b0 2044 2040 2009-03-11T14:47:57Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway) and install a graphical front end: root@desktop:~# apt-get install gnupg gnupg-agent gpgsm kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * Prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] 011c5a77fdcea07fbf3c4382dff0572109886cfb 2045 2044 2009-03-11T14:55:03Z Neil 2 /* GnuPG for Email */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway) and install a graphical front end: root@desktop:~# apt-get install gnupg gnupg-agent gpgsm kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. === Add another identity to a key === === Publish a public key === Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * Prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] 2e9be77038fa59f6721ecfb5265ac0ce759fcf8b 0 1515 2042 1993 2009-03-11T14:27:01Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Crtypography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 0faa17fdc4aee2b553df2f2c50ae0a3aa4df188a 2043 2042 2009-03-11T14:27:20Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 1a19ca920182f1f8b7bd809db8a5b614fc64ec67 0 1579 2049 2048 2009-03-11T20:01:22Z Neil 2 /* Serving pages with Apache and Mongrel cluser */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 + Phusion Passenger == == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] 4a7cdfdfa4227bc2a12a07e25e620d156731a923 2050 2049 2009-03-11T20:13:09Z Neil 2 /* Apache2 + Phusion Passenger */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano == * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] [[Category:Ruby]] [[Category:Rails]] fe2eb77389ddad32e19c76f718d396e088ffd380 2051 2050 2009-03-11T21:06:14Z Neil 2 /* See also */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano == * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] [[Category:Ruby]] [[Category:Rails]] 1ad8fec5ba55ebf7cc13b30f0ba454d1127d5577 2052 2051 2009-03-11T21:11:25Z Neil 2 /* Apache2 and Phusion Passenger, deployed with Capistrano */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano == * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] [[Category:Ruby]] [[Category:Rails]] c10303ff0a6664d671d145eb9d7058959ffaebbb 2053 2052 2009-03-11T21:20:15Z Neil 2 /* Apache2 and Phusion Passenger, deployed with Capistrano */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby == [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2; no Mongrel, no Passenger == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano == * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] [[Category:Ruby]] [[Category:Rails]] 523959be0f53936bf4c4232d08d0322aecb4ba6e 2054 2053 2009-03-11T21:23:08Z Neil 2 wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] [[Category:Ruby]] [[Category:Rails]] 6c8bbc920147c96af8a61ed833c04b3513545546 2055 2054 2009-03-11T21:25:31Z Neil 2 /* See also */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] aaea0434d0b5c11c912d65b04fa9b516230d4d90 2056 2055 2009-03-12T08:32:22Z Neil 2 /* Make Mongrel run as a service */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] cd76818efc117c397fbceef01ec427be09a61426 2057 2056 2009-03-12T08:34:52Z Neil 2 /* Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 09e92e562165ba5f8d39306a8959ee642a078dea 0 1583 2058 2026 2009-03-12T21:02:29Z Neil 2 /* Make orders take real money from PayPal */ wikitext text/x-wiki The Depot application from the [http://www.pragprog.com/titles/rails3/agile-web-development-with-rails-third-edition Agile Web Development with Rails] book is a good way of learning the basics of Rails. But there's nothing like cutting your own groove for really getting to grips with a new technology. In that spirit, here are some extensions to the Depot application that I've made. == Add a date products are available from == (Inspired by a change included in the first edition of the book) The idea is that products will only become available after a certain date. After that date, products are present in the catalogue; before it, they're not listed there. This change was made after the price was added, but before the test data migration was created. If you do this after the test data is created, make sure to include available dates for all the products. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_to_product date_available:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available, :date {{Code highlight}}Product.reset_column_information {{Code highlight}}end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available:</b></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> {{Code highlight}}<nowiki><% if product.date_available.past? %></nowiki> {{Code highlight}}<nowiki><dd>Available since <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> * Include this section in the <code>depot.css</code> stylesheet to make the text stand out when a product us unavailable {{Code highlight}}<nowiki>#product-list .list-description .unavailable {</nowiki> {{Code highlight}}<nowiki>color: #f00;</nowiki> {{Code highlight}}<nowiki>}</nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, {{Code highlight}}:conditions => "date_available <= now()", :order => :title) end The catalogue should now only include products that are marked as being available. == Add a date products are available until == This is much the same as the change above, except that there is the added complication that not every product will have an end-of-availability date. If this date is unknown, it should be stored as <code>nil</code> in the database and the product should remain available forever. === Create a migration to add the 'date available' field === * First, create a migration to add the new field: user@desktop:depot$ ruby script/generate migration add_date_available_until_to_product date_available_until:date * The migration file itself shouldn't need any modification, so apply it: user@desktop:depot$ rake db:migrate * Sorted! If you have problems with the new field being unrecognised when you add test data, include a call to Product.reset_column_information in the migration: def self.up add_column :products, :date_available_until, :date {{Code highlight}}Product.reset_column_information end === Update the Product view forms === The date field needs to be alterable in the View, Create, and Update forms (we'll do the Index in a moment, below). * In <code>app/views/products/show.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><b>Price:</b></nowiki> <nowiki><%=h @product.price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><b>Date available:</b></nowiki> <nowiki><%=h @product.date_available %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><b>Date available until:</b></nowiki> {{Code highlight}}<nowiki><%=h @product.date_available_until %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><%= link_to 'Edit', edit_product_path(@product) %> |</nowiki> <nowiki><%= link_to 'Back', products_path %></nowiki> * In <code>app/views/products/edit.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Update" %></nowiki> <nowiki></p></nowiki> * In <code>app/views/products/new.html.erb</code>, add these lines near the end: <nowiki><p></nowiki> <nowiki><%= f.label :price %><br /></nowiki> <nowiki><%= f.text_field :price %></nowiki> <nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.label :date_available %><br /></nowiki> <nowiki><%= f.date_select :date_available, :order => [:day, :month, :year] %></nowiki> <nowiki></p></nowiki> {{Code highlight}}<nowiki><p></nowiki> {{Code highlight}}<nowiki><%= f.label :date_available_until %><br /></nowiki> {{Code highlight}}<nowiki><%= f.date_select :date_available_until, :order => [:day, :month, :year], :include_blank => true %></nowiki> {{Code highlight}}<nowiki></p></nowiki> <nowiki><p></nowiki> <nowiki><%= f.submit "Create" %></nowiki> <nowiki></p></nowiki> You should now be able to add availability dates to products. === Prettily format the Product index page === The index view needs a bit more work to make it pretty. * Adjust the display to include the <code>date available</code> field. <nowiki><dl></nowiki> <nowiki><dt><%=h product.title %></dt></nowiki> <nowiki><dd><%=h truncate(product.description.gsub(/<.*?>/, '|'), </nowiki> <nowiki>:length => 80) %></dd></nowiki> <nowiki><dd><%=number_to_currency product.price, :unit => "£" %></nowiki> <nowiki><% if product.date_available.past? %></nowiki>< <nowiki><dd>Available since <%= product.date_available %></dd></nowiki> <nowiki><% else %></nowiki> <nowiki><dd class="unavailable">Available from <%= product.date_available %></dd></nowiki> <nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% if not product.date_available_until.nil? %> </nowiki> {{Code highlight}}<nowiki><% if product.date_available_until.future? %></nowiki> {{Code highlight}}<nowiki><dd>Available until <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% else %></nowiki> {{Code highlight}}<nowiki><dd class="unavailable">Unavailable since <%= product.date_available_until %></dd></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki></dl></nowiki> === Update the Product model to include the date constraint === Add the <code>:conditions</code> option to the <code>find_products_for_sale</code> procedure in <code>app/models/product.rb</code> def self.find_products_for_sale find(:all, :conditions => "date_available <= now() {{Code highlight}}and (date_available_until is null or date_available_until >= now())", :order => :title) end The catalogue should now only include products that are marked as being available. == Sort cart items alphabetically == In <code>app/views/store/_cart.html.erb</code>, make the cart items render in order. <%= render(:partial => 'cart_item', :collection => cart.items{{Code highlight}}.sort_by {|item| item.product.title}) %> == Implement the cart as a genuine database model == '''Not done yet''' (Inspired by a comment in the text about the cart) == AJAXify the checkout form == If you've been following along the book, doing all the additional extras, you'll have a fully AJAXified cart. Unfortunately, that means that it won't work properly with the checkout code as presented in the book. To make it work, you need to make the following modifications. * Add the 'Checkout' button to <code>_cart.html.erb</code> <nowiki><div class="cart-title">Your Cart</div></nowiki> <nowiki><table></nowiki> <nowiki><%= render(:partial => 'cart_item', :collection => cart.items.sort_by {|item| item.product.title}) %></nowiki> <nowiki><tr class="total-line"></nowiki> <nowiki><td colspan="2">Total</td></nowiki> <nowiki><td class="total-cell"><%= number_to_currency(cart.total_price, :unit => "£") %></td></nowiki> <nowiki></tr></nowiki> <nowiki></table></nowiki> {{Code highlight}}<nowiki><% form_remote_tag :url => {:action => 'checkout'} do %></nowiki> {{Code highlight}}<nowiki><%= submit_tag "Checkout" %></nowiki> {{Code highlight}}<nowiki><% end %></nowiki> <nowiki><% form_remote_tag :url => {:action => 'empty_cart'} do %></nowiki> <nowiki><%= submit_tag "Empty cart" %></nowiki> <nowiki><% end %></nowiki> * Modify the <code>checkout</code> action in the <code>store_controller</code> to respond properly to an xhr request. def checkout @cart = find_cart if @cart.items.empty? redirect_to_index("Your cart is empty" ) else @order = Order.new {{Code highlight}}respond_to do |format| {{Code highlight}}format.js if request.xhr? {{Code highlight}}format.html {{Code highlight}}end end end The desired action is for the 'Checkout' button to replace the catalogue display with the checkout form. That means we need a way of identifying the catalogue display so it can be replaced and something to replace it with. We also need to ensure that the checkout form display works both with and without JavaScript. * First, identify the main panel where the catalogue display is. Modify <code>app/views/layouts/store.html.erb</code> to wrap the main catalogue display in a <code><nowiki><div></nowiki></code>: ... <nowiki><div id="main"></nowiki> <nowiki><% if flash[:notice] -%></nowiki> <nowiki><div id="notice"><%= flash[:notice] %></div></nowiki> <nowiki><% end -%></nowiki> {{Code highlight}}<nowiki><div id="main_panel"></nowiki> <nowiki><%= yield :layout %></nowiki> {{Code highlight}}<nowiki></div></nowiki> <nowiki></div></nowiki> ... * For the checkout action to render properly when called as an AJAX operation, it needs a partial to render. This partial should contain the whole checkout form, so rename <code>app/views/checkout.html.erb</code> as <code>app/views/_checkout.html.erb</code> * The <code>app/views/checkout.html.erb</code> view needs to render this partial. This means that non-AJAX calls will render it correctly. The whole view should be this one line: <%= render(:partial => "checkout", :object => @order) %> * Create <code>app/views/checkout.js.rsj</code> to handle displaying the checkout form when called by AJAX page.replace_html("main_panel", :partial => "checkout", :object => @order) :This replaces the contents of the main panel in the standard layout with the partial. == Improve the decoupling between cart items and order items == (Based on the [http://pragprog.wikidot.com/pt-e-1 discussion] at the [http://pragprog.wikidot.com/rails-play-time book's wiki site].) As written in the book, the store controller's save order action creates a new order and passes it the cart. The order model then creates a new line item for each item in the cart (requiring that the order model know about the internal structure of the cart) and the creation of the line item is the responsibility of the line item model, which has to know about the internal structure of the cart item. This is rather a lot of coupling. It's better for the order, line item, cart, and cart item models each to know very little about each other. The only place in the application that knows about both carts and orders is the store controller, so this is were the transfer of information from one to the other should take place. * Remove the <code>add_line_items_from_cart(cart)</code> procedure from <code>app/models/line_item.rb</code> * Remove the <code>self.from_cart_item(cart_item)</code> procedure from <code>app/models/line_item.rb</code> * Modify the <code>save_order</code> procedure in <code>app/controllers/store.rb</code>. Remove the call to <code>@order.add_line_items_from_cart(@cart)</code>. Add the order creation code shown below. def save_order @cart = find_cart @order = Order.new(params[:order]) {{Code highlight}}@cart.items.each do |item| {{Code highlight}}li = LineItem.new {{Code highlight}}li.product = item.product {{Code highlight}}li.quantity = item.quantity {{Code highlight}}li.total_price = item.price {{Code highlight}}@order.line_items << li {{Code highlight}}end if @order.save session[:cart] = nil redirect_to_index("Thank you for your order") else render :action => 'checkout' end end == Eliminate old sessions == '''Not done yet''' == Make orders take real money from PayPal == '''Not done yet''' [[Category:Ruby]] [[Category:Rails]] 352d59afa970af009d6b16b8837890da309e6b94 0 1586 2059 2037 2009-03-14T23:51:50Z Neil 2 wikitext text/x-wiki == ''Getting Things Done'' tracker == * Needs tags == Report writing tracker == * Complex data structure * Dashboard view * Control over who gets to see and edit what * Emailed reminders * File upload * PDF formatting of reports == Child progress monitor == * Children, milestones * Historical dimension of attainment: show progress over time == Chldminder invoice system == * Email invoices * Payment integration * Selective viewing * Task requests * Calendar-based view == Forum system == * Forums * User settings * Private messaging * Selective editing of posts * BBCode parsing * Forum subscription == Moodle forum RSS generator == * Receive emails * Generate RSS == Course support system == * Who's doing which project * Who's generated feedback * Marks, comments, etc. == Agile development user story tracker == * Assign priorities * Show how completed * Show tests generated == Games day organiser == * Register attendees * Record games on offer, with times * Allocate players to games * Blog of updates == Name generators === [[Category:Ruby]] [[Category:Rails]] eac78282302e7c92ee79066225516463b0b4b23f 2060 2059 2009-03-16T11:37:22Z Neil 2 /* Name generators = */ wikitext text/x-wiki == ''Getting Things Done'' tracker == * Needs tags == Report writing tracker == * Complex data structure * Dashboard view * Control over who gets to see and edit what * Emailed reminders * File upload * PDF formatting of reports == Child progress monitor == * Children, milestones * Historical dimension of attainment: show progress over time == Chldminder invoice system == * Email invoices * Payment integration * Selective viewing * Task requests * Calendar-based view == Forum system == * Forums * User settings * Private messaging * Selective editing of posts * BBCode parsing * Forum subscription == Moodle forum RSS generator == * Receive emails * Generate RSS == Course support system == * Who's doing which project * Who's generated feedback * Marks, comments, etc. == Agile development user story tracker == * Assign priorities * Show how completed * Show tests generated == Games day organiser == * Register attendees * Record games on offer, with times * Allocate players to games * Blog of updates == Name generators == [[Category:Ruby]] [[Category:Rails]] f007a08e0cda548bf7cf7ea76f5f6e28ab5089fb 2061 2060 2009-03-16T19:55:28Z Neil 2 /* Chldminder invoice system */ wikitext text/x-wiki == ''Getting Things Done'' tracker == * Needs tags == Report writing tracker == * Complex data structure * Dashboard view * Control over who gets to see and edit what * Emailed reminders * File upload * PDF formatting of reports == Child progress monitor == * Children, milestones * Historical dimension of attainment: show progress over time == Chldminder invoice system == * Calculate fees: fixed prices, price per hour, minimum durations (e.g. £3.50 per hour, min 1 hour) * Email and view invoices * Payment integration * Selective viewing: parent to view only their children * Task requests (email forwarding) * Calendar-based view: global and per child/parent * Early learning record ** Day-by-day diary, viewable per week ** include photos == Forum system == * Forums * User settings * Private messaging * Selective editing of posts * BBCode parsing * Forum subscription == Moodle forum RSS generator == * Receive emails * Generate RSS == Course support system == * Who's doing which project * Who's generated feedback * Marks, comments, etc. == Agile development user story tracker == * Assign priorities * Show how completed * Show tests generated == Games day organiser == * Register attendees * Record games on offer, with times * Allocate players to games * Blog of updates == Name generators == [[Category:Ruby]] [[Category:Rails]] 7e382a310d6339fa1c9c02eb687070b08bc2492f 0 1587 2062 2045 2009-03-18T10:37:55Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. === Add another identity to a key === === Publish a public key === Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] 6bb26a7a0fe502a877044774ba0e43d1f465fcc4 0 1578 2063 2029 2009-03-20T15:57:27Z Neil 2 /* Subversion control setup */ wikitext text/x-wiki Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! [[Category:Ruby]] [[Category:Rails]] 22f30639aa21f70fa748deebe0476b7e2716bb36 2064 2063 2009-03-20T16:02:25Z Neil 2 /* Subversion control setup */ wikitext text/x-wiki Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! [[Category:Ruby]] [[Category:Rails]] f189fe8b4a0a57d9dd82a22e39f231983f300a16 2095 2064 2009-05-06T20:39:00Z Neil 2 wikitext text/x-wiki Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] e2e0cf25c9fd680777099143a7250fdf11a25d03 0 1526 2065 2041 2009-03-27T00:31:11Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 30972c0659c3252172a235f89cec3ec3451a5b35 2086 2065 2009-04-30T08:07:59Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] d5d7b5382419a253659dda99c7f68e9f247c390a 2087 2086 2009-05-01T09:31:07Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 919c6b06f5af6e9b7e17291411819c819a3c01d7 0 1543 2066 2030 2009-04-15T20:51:30Z Neil 2 /* Iptables.rules */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> == Protect the SSH port == === Iptables.rules === This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END === Iptables script === #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 062c3c5d1becf6bb62d07b0362b25dfcdd6453df 2067 2066 2009-04-15T20:53:17Z Neil 2 /* Protect the SSH port */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> == Protect the SSH port == == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] f7a786189a5442ed99db6db9a18f6437b492bf36 2069 2067 2009-04-15T21:21:52Z Neil 2 /* Protect the SSH port */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> == Protect the SSH port == If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. Both of the other two items can be managed either in your router's setup or in your computers' firewalls. If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. To change the port used by SSH, you can alter the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Include this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] d0a711c9efa7712aa5f2eee9523dc59abcfd9f98 2070 2069 2009-04-16T07:22:07Z Neil 2 /* Protect the SSH port */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> == Protect the SSH port == If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. Both of the other two items can be managed either in your router's setup or in your computers' firewalls. === Changing the SSH port === If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. If you want to do this with your router, use the routers 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. If you want to do it with your own firewall, open the higher port and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 4e39c3c2c907e49437e218d930f5fb08786e2237 2071 2070 2009-04-21T22:27:44Z Neil 2 Moved SSH protection to a separate page wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] c7a44789bbc1ccd060b21c384d396ae8593529a3 2075 2071 2009-04-21T23:08:56Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect ssh]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 3ebd1a2702e11449304a5555b935bb5f0a68bd8c 2084 2075 2009-04-22T12:17:16Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] e8d8eeb71652a64bdf55f383f4ea973ba0daca27 0 1553 2068 1863 2009-04-15T20:59:04Z Neil 2 /* Port forwarding */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| None |align="center" width="50%"| [[OS installation]] |} The router needs a bit of setting up to allow all the server services to be available. This should be done in stages, as things are installed on the server. This document collects them all together for convenience. == Get it working == The first step is to simply get the router working, so that it connects to your ISP and can handle satellite machines connecting via DHCP. == DHCP ranges == In order for the machines with [[Hostname and IP setup|fixed IP numbers]] to avoid stomping all over the router's DHCP server, you need to ensure that some addresses on the local network are not available for allocation by the router's DHCP server. Configure the router so that addresses from 192.168.1.250 to 192.168.1.254 are outside the DHCP pool. == DNS rerouting == By default, the router uses the ISP's DNS servers to find addresses for its DHCP-connected machines. Unfortunately, doing this means that these machines won't be able to find the addresses of the various machines on the LAN. Once the [[DNS server setup|DNS servers]] are set up, change the router's settings so that it no longer gets DNS server information from the ISP, but uses the nameservers you specify: specify the ones you've just set up. == Port forwarding == To make the various services you've set up available to the outside world, you'll need to pass requests from the outside world into the server. The router should have a 'port forwarding' facility or similar, where incoming traffic to the router on a particular port is passed to a specified IP number and port on the LAN. Once you've set up a service, you can open the port for it. The ports you'll need to open are: * Port 25 for [[Postfix server setup|SMTP mail transport]] * Port 80 for [[Web server setup|serving Web pages]] * Port 443 for secure web pages, such as used by [[Webmail setup|webmail]] You may want to consider opening port 22 (SSH) which will allow secure remote logins to a machine. If you do, you'll probably need to protect it against opportunistic login attempts by passersby. See the [[Firewall_setup#Protect_the_SSH_port|discussion on the firewall page]] for details. [[Category:Server setup]] b96b8f404a951cc6324c9e45dee32daef048de42 0 1588 2072 2009-04-21T23:06:21Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. Both of the other two items can be managed either in your router's setup or in your computers' firewalls. == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict those connections using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] 6126b6483e755f027fd4c112dac0b9e2367e1cfb 2076 2072 2009-04-21T23:09:00Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. Both of the other two items can be managed either in your router's setup or in your computers' firewalls. == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict those connections using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] d624be89e8b1ab70c105f75349100945c73a9e4e 2077 2076 2009-04-21T23:15:32Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. == Changes to config file == But before that, there are a couple of changes that need to be made to the <code>/etc/ssh/sshd_config</code> file # Authentication: PermitRootLogin no to prevent attempts to log in as root. Add the line AllowUsers user1 user2 user3 to only allow SSH access by these users. == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict those connections using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] 4b5765c1a4caf6e67ccc05ca33a576f6220cbaf9 2078 2077 2009-04-21T23:17:21Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. == Changes to config file == But before that, there are a couple of changes that need to be made to the SSH server's <code>/etc/ssh/sshd_config</code> file # Authentication: PermitRootLogin no to prevent attempts to log in as root. Add the line AllowUsers user1 user2 user3 to only allow SSH access by these users. Then restart the SSH server: root@server:~# /etc/init.d/ssh restart == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict those connections using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] c6945c1f8dda21cd6ec65a6a7b83dde9275fc817 2079 2078 2009-04-22T06:42:28Z Neil 2 /* Changes to config file */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. == Changes to config file == But before that, there are a couple of changes that need to be made to the SSH server's <code>/etc/ssh/sshd_config</code> file. Disable root logins by changing the line to this # Authentication: PermitRootLogin no to prevent attempts to log in as root. Add the line AllowUsers user1 user2 user3 to only allow SSH access by these users. Then restart the SSH server: root@server:~# /etc/init.d/ssh restart == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict those connections using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] 199048ad4518a8f5daa3317e7725fcf81f1adb77 2080 2079 2009-04-22T06:49:12Z Neil 2 /* Filter connection attempts in the firewall */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. == Changes to config file == But before that, there are a couple of changes that need to be made to the SSH server's <code>/etc/ssh/sshd_config</code> file. Disable root logins by changing the line to this # Authentication: PermitRootLogin no to prevent attempts to log in as root. Add the line AllowUsers user1 user2 user3 to only allow SSH access by these users. Then restart the SSH server: root@server:~# /etc/init.d/ssh restart == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict connections to those machines using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] c4b9b0568c8ac280c414c57cc4bb78aa5b9d0b7d 2081 2080 2009-04-22T12:16:16Z Neil 2 [[Protect ssh]] moved to [[Protect SSH]] wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. == Changes to config file == But before that, there are a couple of changes that need to be made to the SSH server's <code>/etc/ssh/sshd_config</code> file. Disable root logins by changing the line to this # Authentication: PermitRootLogin no to prevent attempts to log in as root. Add the line AllowUsers user1 user2 user3 to only allow SSH access by these users. Then restart the SSH server: root@server:~# /etc/init.d/ssh restart == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict connections to those machines using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] c4b9b0568c8ac280c414c57cc4bb78aa5b9d0b7d 0 1515 2073 2043 2009-04-21T23:07:43Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect ssh]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect ssh]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] c2b81a37cb0c54d8bde4e685e2b5918b29227bf0 2083 2073 2009-04-22T12:16:56Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 8495635aad1ab6c22ee0899bfe6d99b0c86a8fa0 0 1544 2074 1962 2009-04-21T23:08:52Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[Protect ssh]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld NS desktop.domain.tld ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/bind/run root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run * Modify the file <tt>/etc/default/bind9</tt> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * Modify <tt>/etc/default/syslogd</tt> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * Make AppArmor allow BIND access to the chroot jail. Modify <tt>/etc/apparmor.d/usr.sbin.named</tt> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/run/bind/run/named.pid w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, Sysklogd, and BIND: root@server:~# /etc/init.d/apparmor restart root@server:~# /etc/init.d/sysklogd restart root@server:~# /etc/init.d/bind9 restart * Check <tt>/var/log/syslog</tt> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 084198415d59593a8e3ef1f0b6696550ab1e103f 2085 2074 2009-04-22T12:18:41Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld NS desktop.domain.tld ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/bind/run root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run * Modify the file <tt>/etc/default/bind9</tt> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * Modify <tt>/etc/default/syslogd</tt> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * Make AppArmor allow BIND access to the chroot jail. Modify <tt>/etc/apparmor.d/usr.sbin.named</tt> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/run/bind/run/named.pid w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, Sysklogd, and BIND: root@server:~# /etc/init.d/apparmor restart root@server:~# /etc/init.d/sysklogd restart root@server:~# /etc/init.d/bind9 restart * Check <tt>/var/log/syslog</tt> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] e849e1bbd42e6d1de1d64ca28f8577c84e86ccb0 0 1590 2088 2009-05-01T14:58:04Z Neil 2 New page: Read the tutorial at [http://www.railsforum.com/viewtopic.php?id=14216 RailsForum]. Follow the instructions to download and install the tutorial. But the tutorial system needs ExceptionL... wikitext text/x-wiki Read the tutorial at [http://www.railsforum.com/viewtopic.php?id=14216 RailsForum]. Follow the instructions to download and install the tutorial. But the tutorial system needs ExceptionLogger to be installed. That isn't done in the tutorial as ExceptionLogger still resides in a SVN repository, not on GitHub. So ExceptionLogger needs to be installed manually. ruby script/plugin install http://svn.techno-weenie.net/projects/plugins/exception_logger/ Modify app/application_controller.rb to include the lines include ExceptionLoggable include WillPaginate git submodule init git submodule add git://github.com/defunkt/exception_logger.git vendor/plugins/exception_logger git submodule update [[Category:Ruby]] [[Category:Rails]] 9551180b2a70468f413e5f601cb635134d8584a6 2089 2088 2009-05-06T11:06:10Z Neil 2 wikitext text/x-wiki Read the tutorial at [http://www.railsforum.com/viewtopic.php?id=14216 RailsForum]. Follow the instructions to download and install the tutorial. But the tutorial system needs ExceptionLogger to be installed. That isn't done in the tutorial as ExceptionLogger still resides in a SVN repository, not on GitHub. So ExceptionLogger needs to be installed manually. ruby script/plugin install http://svn.techno-weenie.net/projects/plugins/exception_logger/ Modify app/application_controller.rb to include the lines include ExceptionLoggable include WillPaginate git submodule init git submodule add git://github.com/defunkt/exception_logger.git vendor/plugins/exception_logger git submodule update Modify vendor/plugins/exception_logger/init.rb to uncomment the lines require 'will_paginate' $PAGINATION_TYPE = 'will_paginate' WillPaginate.enable Add this line to config/routes.rb map.connect "logged_exceptions/:action/:id", :controller => "logged_exceptions" [[Category:Ruby]] [[Category:Rails]] ad180a11018338398bbf3c252222b8c075ad7923 2090 2089 2009-05-06T11:43:49Z Neil 2 wikitext text/x-wiki Read the tutorial at [http://www.railsforum.com/viewtopic.php?id=14216 RailsForum]. Follow the instructions to download and install the tutorial. But the tutorial system needs ExceptionLogger to be installed. That isn't done in the tutorial as ExceptionLogger still resides in a SVN repository, not on GitHub. So ExceptionLogger needs to be installed manually. ruby script/plugin install http://svn.techno-weenie.net/projects/plugins/exception_logger/ Modify app/application_controller.rb to include the lines include ExceptionLoggable include WillPaginate git submodule init git submodule add git://github.com/defunkt/exception_logger.git vendor/plugins/exception_logger git submodule update Modify vendor/plugins/exception_logger/init.rb to uncomment the lines require 'will_paginate' $PAGINATION_TYPE = 'will_paginate' WillPaginate.enable Add this line to config/routes.rb # map.connect "logged_exceptions/:action/:id", :controller => "logged_exceptions" map.resources :logged_exceptions [[Category:Ruby]] [[Category:Rails]] 8413487ceb57c065e72bdfa16435db371515bac9 2091 2090 2009-05-06T11:50:41Z Neil 2 wikitext text/x-wiki Read the tutorial at [http://www.railsforum.com/viewtopic.php?id=14216 RailsForum]. Follow the instructions to download and install the tutorial. But the tutorial system needs ExceptionLogger to be installed. Why that isn't done in the tutorial anyway is a mystery to me. So ExceptionLogger needs to be installed manually. git submodule init git submodule add git://github.com/defunkt/exception_logger.git vendor/plugins/exception_logger git submodule update Modify app/application_controller.rb to include the line include ExceptionLoggable Modify vendor/plugins/exception_logger/init.rb to uncomment the lines require 'will_paginate' $PAGINATION_TYPE = 'will_paginate' WillPaginate.enable Add these lines to config/routes.rb map.connect "logged_exceptions/:action/:id", :controller => "logged_exceptions" map.resources :logged_exceptions [[Category:Ruby]] [[Category:Rails]] f86aa2030c39194e510a8e42ee9d5b93f9f6e233 2092 2091 2009-05-06T12:59:09Z Neil 2 wikitext text/x-wiki Read the tutorial at [http://www.railsforum.com/viewtopic.php?id=14216 RailsForum]. Follow the instructions to download and install the tutorial. But the tutorial system needs ExceptionLogger to be installed. Why that isn't done in the tutorial anyway is a mystery to me. So ExceptionLogger needs to be installed manually. git submodule init git submodule add git://github.com/defunkt/exception_logger.git vendor/plugins/exception_logger git submodule update Modify app/application_controller.rb to include the line include ExceptionLoggable Modify vendor/plugins/exception_logger/init.rb to uncomment the lines require 'will_paginate' $PAGINATION_TYPE = 'will_paginate' WillPaginate.enable Add these lines to config/routes.rb map.connect "logged_exceptions/:action/:id", :controller => "logged_exceptions" map.resources :logged_exceptions Finally, restart WEBrick and all should be dandy. [[Category:Ruby]] [[Category:Rails]] 751a003905f5bc19a9724e6d8e8d15c9c4fb18f8 2094 2092 2009-05-06T20:36:52Z Neil 2 wikitext text/x-wiki Read the tutorial at [http://www.railsforum.com/viewtopic.php?id=14216 RailsForum]. Follow the instructions to download and install the tutorial. But the tutorial system needs ExceptionLogger to be installed. Why that isn't done in the tutorial anyway is a mystery to me. So ExceptionLogger needs to be installed manually. git submodule init git submodule add git://github.com/defunkt/exception_logger.git vendor/plugins/exception_logger git submodule update Modify app/application_controller.rb to include the line include ExceptionLoggable Modify vendor/plugins/exception_logger/init.rb to uncomment the lines require 'will_paginate' $PAGINATION_TYPE = 'will_paginate' WillPaginate.enable Add these lines to config/routes.rb map.connect "logged_exceptions/:action/:id", :controller => "logged_exceptions" map.resources :logged_exceptions Finally, restart WEBrick and all should be dandy. == See also == * [http://github.com/defunkt/exception_logger/tree/master ExceptionLogger plugin home] * [http://github.com/mislav/will_paginate/tree/master WillPaginate plugin home] * [http://github.com/technoweenie/restful-authentication/tree/master RestfulAuthentication plugin home] * [http://www.railsforum.com/viewtopic.php?id=14216 RestfulAuthentication + ExceptionLogger + StateMachine +... sample] * [http://avnetlabs.com/rails/restful-authentication-with-rails-2 Installing Restful Authentication with Rails 2] * [http://avnetlabs.com/rails/restful-authentication-with-rails-2-usage Using Restful Authentication with Rails 2] [[Category:Ruby]] [[Category:Rails]] 7fd921e540c0ba51eae7b33bc2d86eb7d8216d24 2096 2094 2009-05-07T06:56:13Z Neil 2 wikitext text/x-wiki Read the tutorial at [http://www.railsforum.com/viewtopic.php?id=14216 RailsForum]. Follow the instructions to download and install the tutorial. But the tutorial system needs ExceptionLogger to be installed. Why that isn't done in the tutorial anyway is a mystery to me. So ExceptionLogger needs to be installed manually. git submodule init git submodule add git://github.com/defunkt/exception_logger.git vendor/plugins/exception_logger git submodule update Modify app/application_controller.rb to include the line include ExceptionLoggable Modify vendor/plugins/exception_logger/init.rb to uncomment the lines require 'will_paginate' $PAGINATION_TYPE = 'will_paginate' WillPaginate.enable Add these lines to config/routes.rb map.connect "logged_exceptions/:action/:id", :controller => "logged_exceptions" map.resources :logged_exceptions Finally, restart WEBrick and all should be dandy. == See also == * [http://github.com/defunkt/exception_logger/tree/master ExceptionLogger plugin home] * [http://github.com/mislav/will_paginate/tree/master WillPaginate plugin home] * [http://github.com/technoweenie/restful-authentication/tree/master RestfulAuthentication plugin home] * [http://www.railsforum.com/viewtopic.php?id=14216 RestfulAuthentication + ExceptionLogger + StateMachine +... sample] * [http://avnetlabs.com/rails/restful-authentication-with-rails-2 Installing Restful Authentication with Rails 2] * [http://avnetlabs.com/rails/restful-authentication-with-rails-2-usage Using Restful Authentication with Rails 2] * [http://snippets.aktagon.com/snippets/78-How-to-install-the-exception-logger-Rails-plugin-and-protect-the-logs-with-basic-authentication Using basic authentication with Exception Logger] [[Category:Ruby]] [[Category:Rails]] 7b5d936fc8b51493a0adf975d563d203e553d51b 0 1591 2093 2009-05-06T13:21:13Z Neil 2 New page: To get the root password, and hence SSH access, to the N800 internet tablet, follow these instructions: [http://internettablettalk.com/forums/showthread.php?t=23932]. Basically: * Install... wikitext text/x-wiki To get the root password, and hence SSH access, to the N800 internet tablet, follow these instructions: [http://internettablettalk.com/forums/showthread.php?t=23932]. Basically: * Install the OpenSSH server * In an X terminal, give the commands: $ sudo gainroot # passwd :and give <code>root</code> a password. * Find the N800's IP number from the window menu (top-left corner) of the Connection Manager. * SSH into it at <code>root@192.168.0.2</code> (or whatever the N800's IP number is) [[Category:Server setup]] dfb24ed97532bf6b161d3bfbf0150e6e722fdebb 0 1540 2097 1867 2009-05-29T08:08:30Z Neil 2 /* Desktop backup */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <tt>/etc/cron.daily/makedailybackups</tt>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -ctime +2 -exec rm -f {} \; # Create file backups tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, adjust the datestamps in the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. [[Category:Server setup]] 769835e1c7f6a5e7a511119e26cdf2c64e0661b5 2098 2097 2009-05-29T08:09:50Z Neil 2 /* Server backup */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, adjust the datestamps in the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. [[Category:Server setup]] ce88f40a0666ab3682771131c37003cca203f81e 0 1536 2099 1856 2009-05-29T10:46:48Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Backup setup|Backup]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <tt>/etc/samba/smb.conf</tt> (note that you'll have to skip around the file to find all the settings) [global] ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of workgroup = myworkgroup ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. unix password sync = no #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart Note that each user will need to log into the Samba server machine directly (i.e. via the GUI at that machine) to go through the 'first login mandatory password reset' rigmarole. Once that's done, they'll be able to use their Samba share. On the Windows side, the user should point Windows Explorer at <code>\\server\username</code> and give the Samba username and password to log in. They can then map a new network drive to the Samba directory if they want. == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. * [http://us6.samba.org/samba/docs/Samba24Hc13.pdf Samba troubleshooting guide] (pdf) [[Category:Server setup]] 98766b38ac536d35b7149b420cb134d8756a011a 0 1515 2100 2083 2009-06-01T11:08:57Z Neil 2 Added to-do list wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) == To do == There are a few other things I need to do. * PGP plugin for SquirrelMail * OpenVPN server * DomainKeys and DKIP data for Postfix [[Category:Server setup]] 6093306a92bd96614cc23c5ab57239b271e58e9c 2101 2100 2009-06-02T19:37:24Z Neil 2 /* To do */ wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) == To do == There are a few other things I need to do. * PGP plugin for SquirrelMail * OpenVPN server * DomainKeys and DKIP data for Postfix * Log viewer for Apache * SSH+SVN access to the Subversion repository * Web-based Git server [[Category:Server setup]] b7a2bb9c5d215ca228fba837680508306b6a44a5 2124 2101 2009-07-24T09:28:53Z Neil 2 /* To do */ Removed section wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] f88ed5bd5103989401b65f358c22f89827411e2c 2149 2124 2009-09-30T09:03:38Z Neil 2 Added Git wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Git setup|Version control system server (Git)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] f601faa41bac5bb91c648b49b6eeeeb4a4992e14 0 1592 2102 2009-07-13T19:41:19Z Neil 2 New page: I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There w... wikitext text/x-wiki I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There were a couple of changes that I needed to do to make it all work. Most of these notes are shamelessly cribbed from the [http://tjn.me/2009/06/install-tracks-17-on-ubuntu/ TJN blog post]. == Create the database and user == As per the [[Ruby on Rails deployment|normal production environment step]], create the necessary database and user. root@server:~# mysql -u root -p mysql> create database tracks; mysql> grant all on tracks.* to 'tracksuser'@'localhost' identified by 'SecretPassword'; == Install the files == * Download Tracks from the [http://www.getontracks.org/ Tracks homepage] and copy the file into <code>/var/www/tracks.domain.tld<code>. * Unzip the files, then change the permissions on them all to 777 root@server:~# cd /var/www/tracks.domain.tld root@server:~# unzip tracks-1.7.zip root@server:~# cd tracks-1.7 root@server:~# chmod -R 777 . root@server:~# chown -R www-data:www-data . :(Some files, such as <code>config/environment.rb</code>, cannot be owned by root, so make the whole lot owned by someone else.) :(The permissions are needed to enable Tracks to create cache files.) == Update the Tracks configuration == * Edit <code>config/database.yml</code> to use the MySQL database: production: adapter: mysql database: tracks host: localhost username: tracksuser password: SecretPassword * Edit <code>config/site.yml</code> to change the password salt: # This is the 'salt' to add to the password before it is encrypted # You need to change this to something unique for yourself # salt: "change-me" salt: "SecretSalt" :(substitute your own salt, of course) == Populate the database == root@server:/var/www/tracks.domain.tld/tracks-1.7# rake db:migrate RAILS_ENV=production == Allow recurrent todos == Modify line 76 of <code>app/views/recurring_todos/_recurring_todo_form.erb</code> to be this: text_field(:recurring_todo, :start_from, "value" => format_date(Time.now), "size" => 12, "class" => "Date", "onfocus" => "Calendar.setup", "tabindex" => 6, "autocomplete" => "off") %><br/> :(instead of a reference to <code>format_date(Time.today)</code> ) == Create the Apache virtual host == * Create the virtual host file, <code>/etc/apache2/sites-available/tracks.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/tracks.domain.tld/tracks-1.7/public ServerName tracks.domain.tld RewriteEngine On RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Remove Tracks's <code>.htaccess</code> file: root@server:var/www/tracks.domain.tld/tracks-1.7# mv public/.htaccess public/dontneedhtaccess * Enable the site: root@server:~# a2ensite tracks.domain.tld root@server:~# /etc/init.d/apache2 reload == Allow restarts of Passenger == * Create the <code>tmp</code> directory: root@server:~# mkdir -p /var/www/tracks.domain.tld/tracks-1.7/tmp * To restart Passenger, issue this command: root@server:~# touch /var/www/tracks.domain.tld/tracks-1.7/tmp/restart.txt [[Category:Rails]] [[Category:Ruby]] a87728325ba41693d39b09607c6f1bd89145a2e4 2114 2102 2009-07-23T20:19:44Z Neil 2 /* Install the files */ wikitext text/x-wiki I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There were a couple of changes that I needed to do to make it all work. Most of these notes are shamelessly cribbed from the [http://tjn.me/2009/06/install-tracks-17-on-ubuntu/ TJN blog post]. == Create the database and user == As per the [[Ruby on Rails deployment|normal production environment step]], create the necessary database and user. root@server:~# mysql -u root -p mysql> create database tracks; mysql> grant all on tracks.* to 'tracksuser'@'localhost' identified by 'SecretPassword'; == Install the files == * Download Tracks from the [http://www.getontracks.org/ Tracks homepage] and copy the file into <code>/var/www/tracks.domain.tld</code>. * Unzip the files, then change the permissions on them all to 777 root@server:~# cd /var/www/tracks.domain.tld root@server:~# unzip tracks-1.7.zip root@server:~# cd tracks-1.7 root@server:~# chmod -R 777 . root@server:~# chown -R www-data:www-data . :(Some files, such as <code>config/environment.rb</code>, cannot be owned by root, so make the whole lot owned by someone else.) :(The permissions are needed to enable Tracks to create cache files.) == Update the Tracks configuration == * Edit <code>config/database.yml</code> to use the MySQL database: production: adapter: mysql database: tracks host: localhost username: tracksuser password: SecretPassword * Edit <code>config/site.yml</code> to change the password salt: # This is the 'salt' to add to the password before it is encrypted # You need to change this to something unique for yourself # salt: "change-me" salt: "SecretSalt" :(substitute your own salt, of course) == Populate the database == root@server:/var/www/tracks.domain.tld/tracks-1.7# rake db:migrate RAILS_ENV=production == Allow recurrent todos == Modify line 76 of <code>app/views/recurring_todos/_recurring_todo_form.erb</code> to be this: text_field(:recurring_todo, :start_from, "value" => format_date(Time.now), "size" => 12, "class" => "Date", "onfocus" => "Calendar.setup", "tabindex" => 6, "autocomplete" => "off") %><br/> :(instead of a reference to <code>format_date(Time.today)</code> ) == Create the Apache virtual host == * Create the virtual host file, <code>/etc/apache2/sites-available/tracks.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/tracks.domain.tld/tracks-1.7/public ServerName tracks.domain.tld RewriteEngine On RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Remove Tracks's <code>.htaccess</code> file: root@server:var/www/tracks.domain.tld/tracks-1.7# mv public/.htaccess public/dontneedhtaccess * Enable the site: root@server:~# a2ensite tracks.domain.tld root@server:~# /etc/init.d/apache2 reload == Allow restarts of Passenger == * Create the <code>tmp</code> directory: root@server:~# mkdir -p /var/www/tracks.domain.tld/tracks-1.7/tmp * To restart Passenger, issue this command: root@server:~# touch /var/www/tracks.domain.tld/tracks-1.7/tmp/restart.txt [[Category:Rails]] [[Category:Ruby]] fa510016b14f94844f1ce2e2ed9ff6771caf917e 2116 2114 2009-07-23T20:52:21Z Neil 2 /* Create the database and user */ wikitext text/x-wiki I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There were a couple of changes that I needed to do to make it all work. Most of these notes are shamelessly cribbed from the [http://tjn.me/2009/06/install-tracks-17-on-ubuntu/ TJN blog post]. == Create the database and user == * As per the [[Ruby on Rails deployment|normal production environment step]], create the necessary database and user. root@server:~# mysql -u root -p mysql> create database tracks; mysql> grant all on tracks.* to 'tracksuser'@'localhost' identified by 'SecretPassword'; mysql> quit; * We can also create a user that Tracks runs as root@server:~# adduser tracks == Install the files == * Download Tracks from the [http://www.getontracks.org/ Tracks homepage] and copy the file into <code>/var/www/tracks.domain.tld</code>. * Unzip the files, then change the permissions on them all to 777 root@server:~# cd /var/www/tracks.domain.tld root@server:~# unzip tracks-1.7.zip root@server:~# cd tracks-1.7 root@server:~# chmod -R 777 . root@server:~# chown -R www-data:www-data . :(Some files, such as <code>config/environment.rb</code>, cannot be owned by root, so make the whole lot owned by someone else.) :(The permissions are needed to enable Tracks to create cache files.) == Update the Tracks configuration == * Edit <code>config/database.yml</code> to use the MySQL database: production: adapter: mysql database: tracks host: localhost username: tracksuser password: SecretPassword * Edit <code>config/site.yml</code> to change the password salt: # This is the 'salt' to add to the password before it is encrypted # You need to change this to something unique for yourself # salt: "change-me" salt: "SecretSalt" :(substitute your own salt, of course) == Populate the database == root@server:/var/www/tracks.domain.tld/tracks-1.7# rake db:migrate RAILS_ENV=production == Allow recurrent todos == Modify line 76 of <code>app/views/recurring_todos/_recurring_todo_form.erb</code> to be this: text_field(:recurring_todo, :start_from, "value" => format_date(Time.now), "size" => 12, "class" => "Date", "onfocus" => "Calendar.setup", "tabindex" => 6, "autocomplete" => "off") %><br/> :(instead of a reference to <code>format_date(Time.today)</code> ) == Create the Apache virtual host == * Create the virtual host file, <code>/etc/apache2/sites-available/tracks.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/tracks.domain.tld/tracks-1.7/public ServerName tracks.domain.tld RewriteEngine On RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Remove Tracks's <code>.htaccess</code> file: root@server:var/www/tracks.domain.tld/tracks-1.7# mv public/.htaccess public/dontneedhtaccess * Enable the site: root@server:~# a2ensite tracks.domain.tld root@server:~# /etc/init.d/apache2 reload == Allow restarts of Passenger == * Create the <code>tmp</code> directory: root@server:~# mkdir -p /var/www/tracks.domain.tld/tracks-1.7/tmp * To restart Passenger, issue this command: root@server:~# touch /var/www/tracks.domain.tld/tracks-1.7/tmp/restart.txt [[Category:Rails]] [[Category:Ruby]] 8a66bfca520cd475b55b3e1628f5083c6136d9ef 2117 2116 2009-07-23T20:56:04Z Neil 2 /* Install the files */ wikitext text/x-wiki I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There were a couple of changes that I needed to do to make it all work. Most of these notes are shamelessly cribbed from the [http://tjn.me/2009/06/install-tracks-17-on-ubuntu/ TJN blog post]. == Create the database and user == * As per the [[Ruby on Rails deployment|normal production environment step]], create the necessary database and user. root@server:~# mysql -u root -p mysql> create database tracks; mysql> grant all on tracks.* to 'tracksuser'@'localhost' identified by 'SecretPassword'; mysql> quit; * We can also create a user that Tracks runs as root@server:~# adduser tracks == Install the files == * Download Tracks from the [http://www.getontracks.org/ Tracks homepage] and copy the file into <code>/var/www/tracks.domain.tld</code>. * Unzip the files root@server:~# cd /var/www/tracks.domain.tld root@server:/var/www/tracks.domain.tld# unzip tracks-1.7.zip root@server:/var/www/tracks.domain.tld# cd tracks-1.7 * Now, fix the ownership and permissions of the files root@server:/var/www/tracks.domain.tld/tracks-1.7# chown -R tracks:tracks * root@server:/var/www/tracks.domain.tld/tracks-1.7# chmod -R 664 * root@server:/var/www/tracks.domain.tld/tracks-1.7# find -type d -exec chmod a+x '{}' \; == Update the Tracks configuration == * Edit <code>config/database.yml</code> to use the MySQL database: production: adapter: mysql database: tracks host: localhost username: tracksuser password: SecretPassword * Edit <code>config/site.yml</code> to change the password salt: # This is the 'salt' to add to the password before it is encrypted # You need to change this to something unique for yourself # salt: "change-me" salt: "SecretSalt" :(substitute your own salt, of course) == Populate the database == root@server:/var/www/tracks.domain.tld/tracks-1.7# rake db:migrate RAILS_ENV=production == Allow recurrent todos == Modify line 76 of <code>app/views/recurring_todos/_recurring_todo_form.erb</code> to be this: text_field(:recurring_todo, :start_from, "value" => format_date(Time.now), "size" => 12, "class" => "Date", "onfocus" => "Calendar.setup", "tabindex" => 6, "autocomplete" => "off") %><br/> :(instead of a reference to <code>format_date(Time.today)</code> ) == Create the Apache virtual host == * Create the virtual host file, <code>/etc/apache2/sites-available/tracks.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/tracks.domain.tld/tracks-1.7/public ServerName tracks.domain.tld RewriteEngine On RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Remove Tracks's <code>.htaccess</code> file: root@server:var/www/tracks.domain.tld/tracks-1.7# mv public/.htaccess public/dontneedhtaccess * Enable the site: root@server:~# a2ensite tracks.domain.tld root@server:~# /etc/init.d/apache2 reload == Allow restarts of Passenger == * Create the <code>tmp</code> directory: root@server:~# mkdir -p /var/www/tracks.domain.tld/tracks-1.7/tmp * To restart Passenger, issue this command: root@server:~# touch /var/www/tracks.domain.tld/tracks-1.7/tmp/restart.txt [[Category:Rails]] [[Category:Ruby]] 726c0d13d7e74f566cef31724d280214d5d7ffad 2119 2117 2009-07-23T21:06:47Z Neil 2 Added link to Rails installation page wikitext text/x-wiki I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There were a couple of changes that I needed to do to make it all work. Most of these notes are shamelessly cribbed from the [http://tjn.me/2009/06/install-tracks-17-on-ubuntu/ TJN blog post]. == Install Rails on the server == This process is described on a separate [[Ruby on Rails installation]] page. == Create the database and user == * As per the [[Ruby on Rails deployment|normal production environment step]], create the necessary database and user. root@server:~# mysql -u root -p mysql> create database tracks; mysql> grant all on tracks.* to 'tracksuser'@'localhost' identified by 'SecretPassword'; mysql> quit; * We can also create a user that Tracks runs as root@server:~# adduser tracks == Install the files == * Download Tracks from the [http://www.getontracks.org/ Tracks homepage] and copy the file into <code>/var/www/tracks.domain.tld</code>. * Unzip the files root@server:~# cd /var/www/tracks.domain.tld root@server:/var/www/tracks.domain.tld# unzip tracks-1.7.zip root@server:/var/www/tracks.domain.tld# cd tracks-1.7 * Now, fix the ownership and permissions of the files root@server:/var/www/tracks.domain.tld/tracks-1.7# chown -R tracks:tracks * root@server:/var/www/tracks.domain.tld/tracks-1.7# chmod -R 664 * root@server:/var/www/tracks.domain.tld/tracks-1.7# find -type d -exec chmod a+x '{}' \; == Update the Tracks configuration == * Edit <code>config/database.yml</code> to use the MySQL database: production: adapter: mysql database: tracks host: localhost username: tracksuser password: SecretPassword * Edit <code>config/site.yml</code> to change the password salt: # This is the 'salt' to add to the password before it is encrypted # You need to change this to something unique for yourself # salt: "change-me" salt: "SecretSalt" :(substitute your own salt, of course) == Populate the database == root@server:/var/www/tracks.domain.tld/tracks-1.7# rake db:migrate RAILS_ENV=production == Allow recurrent todos == Modify line 76 of <code>app/views/recurring_todos/_recurring_todo_form.erb</code> to be this: text_field(:recurring_todo, :start_from, "value" => format_date(Time.now), "size" => 12, "class" => "Date", "onfocus" => "Calendar.setup", "tabindex" => 6, "autocomplete" => "off") %><br/> :(instead of a reference to <code>format_date(Time.today)</code> ) == Create the Apache virtual host == * Create the virtual host file, <code>/etc/apache2/sites-available/tracks.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/tracks.domain.tld/tracks-1.7/public ServerName tracks.domain.tld RewriteEngine On RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Remove Tracks's <code>.htaccess</code> file: root@server:var/www/tracks.domain.tld/tracks-1.7# mv public/.htaccess public/dontneedhtaccess * Enable the site: root@server:~# a2ensite tracks.domain.tld root@server:~# /etc/init.d/apache2 reload == Allow restarts of Passenger == * Create the <code>tmp</code> directory: root@server:~# mkdir -p /var/www/tracks.domain.tld/tracks-1.7/tmp * To restart Passenger, issue this command: root@server:~# touch /var/www/tracks.domain.tld/tracks-1.7/tmp/restart.txt [[Category:Rails]] [[Category:Ruby]] 3348d9764ebcfa073009b403a1c4c9dccf4510c8 2122 2119 2009-07-23T23:04:37Z Neil 2 /* Install the files */ wikitext text/x-wiki I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There were a couple of changes that I needed to do to make it all work. Most of these notes are shamelessly cribbed from the [http://tjn.me/2009/06/install-tracks-17-on-ubuntu/ TJN blog post]. == Install Rails on the server == This process is described on a separate [[Ruby on Rails installation]] page. == Create the database and user == * As per the [[Ruby on Rails deployment|normal production environment step]], create the necessary database and user. root@server:~# mysql -u root -p mysql> create database tracks; mysql> grant all on tracks.* to 'tracksuser'@'localhost' identified by 'SecretPassword'; mysql> quit; * We can also create a user that Tracks runs as root@server:~# adduser tracks == Install the files == * Download Tracks from the [http://www.getontracks.org/ Tracks homepage] and copy the file into <code>/var/www/tracks.domain.tld</code>. * Unzip the files root@server:~# cd /var/www/tracks.domain.tld root@server:/var/www/tracks.domain.tld# unzip tracks-1.7.zip root@server:/var/www/tracks.domain.tld# cd tracks-1.7 * Now, fix the ownership and permissions of the files root@server:/var/www/tracks.domain.tld/tracks-1.7# chown -R tracks:tracks * root@server:/var/www/tracks.domain.tld/tracks-1.7# chmod -R 664 * root@server:/var/www/tracks.domain.tld/tracks-1.7# find -type d -exec chmod a+x '{}' \; root@server:/var/www/tracks.domain.tld/tracks-1.7# chmod -R 664 script/* == Update the Tracks configuration == * Edit <code>config/database.yml</code> to use the MySQL database: production: adapter: mysql database: tracks host: localhost username: tracksuser password: SecretPassword * Edit <code>config/site.yml</code> to change the password salt: # This is the 'salt' to add to the password before it is encrypted # You need to change this to something unique for yourself # salt: "change-me" salt: "SecretSalt" :(substitute your own salt, of course) == Populate the database == root@server:/var/www/tracks.domain.tld/tracks-1.7# rake db:migrate RAILS_ENV=production == Allow recurrent todos == Modify line 76 of <code>app/views/recurring_todos/_recurring_todo_form.erb</code> to be this: text_field(:recurring_todo, :start_from, "value" => format_date(Time.now), "size" => 12, "class" => "Date", "onfocus" => "Calendar.setup", "tabindex" => 6, "autocomplete" => "off") %><br/> :(instead of a reference to <code>format_date(Time.today)</code> ) == Create the Apache virtual host == * Create the virtual host file, <code>/etc/apache2/sites-available/tracks.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/tracks.domain.tld/tracks-1.7/public ServerName tracks.domain.tld RewriteEngine On RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Remove Tracks's <code>.htaccess</code> file: root@server:var/www/tracks.domain.tld/tracks-1.7# mv public/.htaccess public/dontneedhtaccess * Enable the site: root@server:~# a2ensite tracks.domain.tld root@server:~# /etc/init.d/apache2 reload == Allow restarts of Passenger == * Create the <code>tmp</code> directory: root@server:~# mkdir -p /var/www/tracks.domain.tld/tracks-1.7/tmp * To restart Passenger, issue this command: root@server:~# touch /var/www/tracks.domain.tld/tracks-1.7/tmp/restart.txt [[Category:Rails]] [[Category:Ruby]] cb703b1215c34596b7e6d0c84b87b71398c23ddb 2123 2122 2009-07-23T23:04:56Z Neil 2 /* Install the files */ wikitext text/x-wiki I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There were a couple of changes that I needed to do to make it all work. Most of these notes are shamelessly cribbed from the [http://tjn.me/2009/06/install-tracks-17-on-ubuntu/ TJN blog post]. == Install Rails on the server == This process is described on a separate [[Ruby on Rails installation]] page. == Create the database and user == * As per the [[Ruby on Rails deployment|normal production environment step]], create the necessary database and user. root@server:~# mysql -u root -p mysql> create database tracks; mysql> grant all on tracks.* to 'tracksuser'@'localhost' identified by 'SecretPassword'; mysql> quit; * We can also create a user that Tracks runs as root@server:~# adduser tracks == Install the files == * Download Tracks from the [http://www.getontracks.org/ Tracks homepage] and copy the file into <code>/var/www/tracks.domain.tld</code>. * Unzip the files root@server:~# cd /var/www/tracks.domain.tld root@server:/var/www/tracks.domain.tld# unzip tracks-1.7.zip root@server:/var/www/tracks.domain.tld# cd tracks-1.7 * Now, fix the ownership and permissions of the files root@server:/var/www/tracks.domain.tld/tracks-1.7# chown -R tracks:tracks * root@server:/var/www/tracks.domain.tld/tracks-1.7# chmod -R 664 * root@server:/var/www/tracks.domain.tld/tracks-1.7# find -type d -exec chmod a+x '{}' \; root@server:/var/www/tracks.domain.tld/tracks-1.7# chmod -R a+x script/* == Update the Tracks configuration == * Edit <code>config/database.yml</code> to use the MySQL database: production: adapter: mysql database: tracks host: localhost username: tracksuser password: SecretPassword * Edit <code>config/site.yml</code> to change the password salt: # This is the 'salt' to add to the password before it is encrypted # You need to change this to something unique for yourself # salt: "change-me" salt: "SecretSalt" :(substitute your own salt, of course) == Populate the database == root@server:/var/www/tracks.domain.tld/tracks-1.7# rake db:migrate RAILS_ENV=production == Allow recurrent todos == Modify line 76 of <code>app/views/recurring_todos/_recurring_todo_form.erb</code> to be this: text_field(:recurring_todo, :start_from, "value" => format_date(Time.now), "size" => 12, "class" => "Date", "onfocus" => "Calendar.setup", "tabindex" => 6, "autocomplete" => "off") %><br/> :(instead of a reference to <code>format_date(Time.today)</code> ) == Create the Apache virtual host == * Create the virtual host file, <code>/etc/apache2/sites-available/tracks.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/tracks.domain.tld/tracks-1.7/public ServerName tracks.domain.tld RewriteEngine On RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Remove Tracks's <code>.htaccess</code> file: root@server:var/www/tracks.domain.tld/tracks-1.7# mv public/.htaccess public/dontneedhtaccess * Enable the site: root@server:~# a2ensite tracks.domain.tld root@server:~# /etc/init.d/apache2 reload == Allow restarts of Passenger == * Create the <code>tmp</code> directory: root@server:~# mkdir -p /var/www/tracks.domain.tld/tracks-1.7/tmp * To restart Passenger, issue this command: root@server:~# touch /var/www/tracks.domain.tld/tracks-1.7/tmp/restart.txt [[Category:Rails]] [[Category:Ruby]] 1ed6b26186b5ad0f6f0eec63f88cb1a6fef54adc 0 1578 2103 2095 2009-07-17T09:41:42Z Neil 2 /* Development and test databases */ wikitext text/x-wiki Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] 668d7d9599ba57cd26595690f4eb656e849a64f3 2118 2103 2009-07-23T21:03:00Z Neil 2 /* Change the welcome page */ Added log rotation note wikitext text/x-wiki Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: :Rails assumes the user is on the localhost anyway. === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] 295f4c5b29e7f1766b1e6ba7ff299ffbf264bda7 2125 2118 2009-07-29T15:11:24Z Neil 2 /* Development and test databases */ Updated for Sqlite3 and Red Hat wikitext text/x-wiki Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] 7feee94109e05dcdafb5e1a4d76350892438d096 0 1579 2104 2057 2009-07-22T12:55:33Z Neil 2 /* Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host */ Added SVN command for creating a stable branch wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] a45363d3d96b0f4b5f35a87bc62651da635ba3a6 2105 2104 2009-07-22T12:56:11Z Neil 2 /* Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 9601eae7dbbcd4ff24a364720bbd47b3395bda3a 2106 2105 2009-07-22T12:57:32Z Neil 2 /* Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 5ca0f29c7e7491fd16c66c58fb350d5bcb55c460 2107 2106 2009-07-22T13:17:04Z Neil 2 /* Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 9601eae7dbbcd4ff24a364720bbd47b3395bda3a 2108 2107 2009-07-22T15:25:20Z Neil 2 /* Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. * Install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * Replace the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> with one containing just this <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@desktop:~# /etc/init.d/apache2 restart :and your site should be up, assuming it's already deployed there. * For deployment, you need to modify the Capistrano deployment recipe. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end :You should now be able to deploy a new version of your app with <code>cap deploy</code> and <code>cap deploy:migrations</code> as usual. == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 7c84eca5da299cbb7299440c2451b355fe0bc16d 2115 2108 2009-07-23T20:31:05Z Neil 2 /* Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host */ Made instructions fuller wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] ac9d31b238b74b5107423a7b047525a0b0c0ca18 0 1 2109 1823 2009-07-23T20:16:36Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. I also have a few [[downloadable programs]] (nothing exciting). == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 74790d27ed498a0a7f010c84cbcafeb3faa7ea37 14 1580 2110 1983 2009-07-23T20:18:01Z Neil 2 wikitext text/x-wiki See the [http://ruby-doc.org/ Ruby documentation] and [http://api.rubyonrails.org/ Rails documentation]. [[Category:Programming]] d842d3067c76383316720998f9e524053d2d1f3f 2113 2110 2009-07-23T20:19:24Z Neil 2 wikitext text/x-wiki See the [http://ruby-doc.org/ Ruby documentation], [http://api.rubyonrails.org/ Rails documentation], and [[:Category:Ruby|Ruby pages here]]. [[Category:Programming]] a7802da305fa500e82e82dfacbdb5f4603459639 14 1581 2111 1984 2009-07-23T20:18:22Z Neil 2 wikitext text/x-wiki See the [http://ruby-doc.org/ Ruby documentation] [[Category:Programming]] 7ea369ce15feab873c2ecd0939ebaeca153179b3 2112 2111 2009-07-23T20:18:48Z Neil 2 wikitext text/x-wiki See the [http://ruby-doc.org/ Ruby documentation] and [[:Category:Rails|Rails pages]]. [[Category:Programming]] 9997ec59019a05c30653d19f37883aecaa0ea36b 0 1577 2120 2047 2009-07-23T21:09:16Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby :(untested by me) * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] d3c1f4af2474ec21c930eaad992340fbcebc2777 2121 2120 2009-07-23T21:09:47Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby :(untested by me) * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem --version ** Check you're at version 1.3.1 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.2.2 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 17af09222bc7250e2fca6db0edaec4b3e9304f2b 0 1526 2126 2087 2009-09-09T08:31:18Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 0dffb98f84c6ccb7c1f987ccd9ede40b8e78146f 2127 2126 2009-09-09T08:31:36Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server<code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 5f9701b004736157135342a4f192935631284629 2128 2127 2009-09-25T12:34:25Z Neil 2 /* Password-less SSH logins */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. [[Category:Server setup]] 263acbd03f54feb87d62a6bfeaca5f6502843f61 0 1588 2129 2081 2009-09-25T14:06:54Z Neil 2 /* Changes to config file */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. == Changes to config file == But before that, there are a couple of changes that need to be made to the SSH server's <code>/etc/ssh/sshd_config</code> file. Disable root logins by changing the line to this # Authentication: PermitRootLogin no to prevent attempts to log in as root. Add the line AllowUsers user1 user2 user3 to only allow SSH access to these users. Then restart the SSH server: root@server:~# /etc/init.d/ssh restart == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict connections to those machines using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] a6a2fc006816af5641f8e31c1a21e5e45e2fbe2a 0 1593 2130 2009-09-25T14:54:30Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# mkdir gitosis && cd gitosis root@server:~/gitosis# git clone git://eagain.net/gitosis.git root@server:~/gitosis# cd gitosis root@server:~/gitosis/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop~:$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * On the desktop machine, checkout the Gitosis repository user@desktop~:$ git clone git@server:gitosis-admin.git user@desktop~:$ cd gitosis-admin == See also == * [[Category:Server setup]] 516f441b1ea56723c41c48d9664d4256200dd43c 2131 2130 2009-09-25T15:31:48Z Neil 2 /* Set up Gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop~:$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop~:$ git clone git@server:gitosis-admin.git user@desktop~:$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below). == See also == * [[Category:Server setup]] f72fa0691da30b34c66274c1a94e7a1701fb3d09 2132 2131 2009-09-25T15:32:56Z Neil 2 /* Set up Gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop~:$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop~:$ git clone git@server:gitosis-admin.git user@desktop~:$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop~/gitosis-admin:$ git commit -a -m "Some commit message" user@desktop~/gitosis-admin:$ git push == See also == * [[Category:Server setup]] fba61dc40290a7a24b7a849f2db1e46e126efa63 2133 2132 2009-09-25T15:35:58Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop~:$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop~:$ git clone git@server:gitosis-admin.git user@desktop~:$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop~/gitosis-admin:$ git commit -a -m "Some commit message" user@desktop~/gitosis-admin:$ git push == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] 9a8e8ca082e5ea228f59108b91cc9ebb8221a87b 2134 2133 2009-09-25T15:47:45Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up gitweb == == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. == Creating new users in gitosis == == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] 62270617d7ce53f2db0f338ba4d3010b125798c2 2136 2134 2009-09-25T16:07:49Z Neil 2 /* Set up gitweb */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. == Creating new users in gitosis == == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] 7b718d32aa8c1b7811d64dba852b593033a5d1c0 2137 2136 2009-09-28T10:51:31Z Neil 2 /* Set up gitweb */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. == Creating new users in gitosis == == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] 2ade77549993ad796e9ba286853368033202d871 2138 2137 2009-09-28T10:52:38Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. == Creating new users in gitosis == == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] 73d9072e9e508ca2049270af7bc6328074a1ed52 2139 2138 2009-09-28T11:03:11Z Neil 2 /* Creating new users in gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] b44ebe0b221e11a6598cedec9ae563ae44509941 2140 2139 2009-09-28T11:05:19Z Neil 2 /* Creating new users in gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository == * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. == Creating new users in gitosis (untested) == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] 267c3686f1f9f49393712f45fc20a4115507c337 2141 2140 2009-09-28T11:06:23Z Neil 2 /* Set up a basic Git repository */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. == Creating new users in gitosis (untested) == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] d60164311969ff214bd324a83da9bf22d61e6061 2142 2141 2009-09-28T15:04:49Z Neil 2 /* Creating new repositories in gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis (untested) == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] [[Category:Server setup]] 55bfbe2d6964677ff55673d10d48c53a801cd9bc 2143 2142 2009-09-28T15:08:06Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis (untested) == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] [[Category:Server setup]] 9efc82b9f7a28401df545328d5bb87bbb7ec54d9 2144 2143 2009-09-28T15:35:04Z Neil 2 /* Sample <code>gitosis.conf</code> file */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis (untested) == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 http://site.domain.tld/svn/svn_repo user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] [[Category:Server setup]] a13b9197d028b60c572a61511f58679474ba4023 2145 2144 2009-09-28T15:35:21Z Neil 2 /* Converting a Subversion repository to Git */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis (untested) == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 http://site.domain.tld/svn/svn_repo user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] [[Category:Server setup]] 65f2d973408369b6bcf6c41d6d9ce93ad9aa4391 2146 2145 2009-09-28T15:35:40Z Neil 2 /* Converting a Subversion repository to Git */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis (untested) == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] [[Category:Server setup]] f55741eb3415c6766313f21dcc19f943bb4ef762 2147 2146 2009-09-28T20:50:22Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis (untested) == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 20d8609968f1368b7d1612b37ab62a1ee7c50172 2148 2147 2009-09-30T08:59:43Z Neil 2 /* Creating new users in gitosis (untested) */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 3b99bf59cecf4c3818396ca1d363d3296b42e192 0 1543 2135 2084 2009-09-25T16:07:11Z Neil 2 /* Iptables.rules */ Added git line wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables defaults 13 Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <tt>/etc/services</tt> == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 761717e8ab14954681985050c2b4c6ebab733c5b 0 1536 2150 2099 2009-09-30T09:06:08Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Git setup|Git]] |align="center" width="50%"| [[Backup setup|Backup]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <tt>/etc/samba/smb.conf</tt> (note that you'll have to skip around the file to find all the settings) [global] ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of workgroup = myworkgroup ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. unix password sync = no #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart Note that each user will need to log into the Samba server machine directly (i.e. via the GUI at that machine) to go through the 'first login mandatory password reset' rigmarole. Once that's done, they'll be able to use their Samba share. On the Windows side, the user should point Windows Explorer at <code>\\server\username</code> and give the Samba username and password to log in. They can then map a new network drive to the Samba directory if they want. == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. * [http://us6.samba.org/samba/docs/Samba24Hc13.pdf Samba troubleshooting guide] (pdf) [[Category:Server setup]] 3feda3d9637f8e32a07a362aa0bc60cce0e3e5d5 2167 2150 2009-11-30T23:20:14Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Git setup|Git]] |align="center" width="50%"| [[Log viewwer setup|Log stats]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <tt>/etc/samba/smb.conf</tt> (note that you'll have to skip around the file to find all the settings) [global] ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of workgroup = myworkgroup ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. unix password sync = no #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart Note that each user will need to log into the Samba server machine directly (i.e. via the GUI at that machine) to go through the 'first login mandatory password reset' rigmarole. Once that's done, they'll be able to use their Samba share. On the Windows side, the user should point Windows Explorer at <code>\\server\username</code> and give the Samba username and password to log in. They can then map a new network drive to the Samba directory if they want. == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. * [http://us6.samba.org/samba/docs/Samba24Hc13.pdf Samba troubleshooting guide] (pdf) [[Category:Server setup]] cd40c97e33c0db53388cfadc1d4b1e1ae0a9aa4e 2169 2167 2009-11-30T23:21:07Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Git setup|Git]] |align="center" width="50%"| [[Log viewer setup|Log stats]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <tt>/etc/samba/smb.conf</tt> (note that you'll have to skip around the file to find all the settings) [global] ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of workgroup = myworkgroup ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. unix password sync = no #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart Note that each user will need to log into the Samba server machine directly (i.e. via the GUI at that machine) to go through the 'first login mandatory password reset' rigmarole. Once that's done, they'll be able to use their Samba share. On the Windows side, the user should point Windows Explorer at <code>\\server\username</code> and give the Samba username and password to log in. They can then map a new network drive to the Samba directory if they want. == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. * [http://us6.samba.org/samba/docs/Samba24Hc13.pdf Samba troubleshooting guide] (pdf) [[Category:Server setup]] 9c12a18a47dd4bc32836c252370501d697d12306 0 1593 2152 2148 2009-09-30T09:06:17Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit </code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 2f05d49bb6a2e31cee8bcda3236210930d5b6457 0 1577 2153 2121 2009-10-05T13:12:37Z Neil 2 /* Installing Rails with NetBeans 6.5 on Windows */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient15-dev libruby-extras \ libyaml-ruby libzlib-ruby \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient15-dev \ libyaml-ruby libzlib-ruby libruby-extras \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * If you want SQLite3 support, install the packages and the gem: root@desktop:~# apt-get install libsqlite3-0 libsqlite3-dev root@desktop:~# gem install sqlite3-ruby * If you want Ruby support for HTTPS, install the package root@desktop:~# apt-get install libopenssl-ruby :(untested by me) * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 6ce865a0983af9698666ea1c5e346d55a88d49b1 0 1531 2154 1761 2009-10-05T17:31:13Z Neil 2 /* Basic Postfix with virtual users */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki.dovecot.org Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki.dovecot.org/LDA Deliver] local delivery agent, to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Postfix, TLS, and the Amavis + SpamAssassin + ClamAV are here. Dovecot, Deliver, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Basic Postfix with virtual users == Here we get Postfix running, by sending mail to other sites, and receiving mail and putting it in virtual users' mailboxes. I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /home/vmail/domain1.com/user1/ /user2/ /domain2.com/user1/ /user2/ /domain3.com/user1/ ...and so on. The <tt>vmail</tt> user will be the one used for email transport into and out of these virtual mailboxes. * Install postfix root@server:~# apt-get install postfix mailx * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update <tt>/etc/postfix/main.cf</tt> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <tt>/etc/postfix/vhosts</tt> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <tt>/etc/postfix/vmaps</tt> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <tt>/etc/postfix/vmaps</tt> is changed) * Create aliases for the users in <tt>/etc/postfix/valiases</tt> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <tt>/etc/postfix/valiases</tt> is changed) * Create <tt>/etc/aliases</tt> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <tt>telnet</tt> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <tt>mail</tt> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <tt>/etc/postfix/main.cf</tt> file to point to the certificates # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <tt>quit</tt> to end the session. SASL is mentioned in various places, but it isn't needed. SASL allows access to the SMTP server from remote sites. This Postfix installation is configured so that only machines on my LAN can use it. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <tt>/etc/mailname</tt> mydomain.com * Install the packages: root@server:~# apt-get install amavis-new spamassassin clamav === ClamAV === * Add the ClamAV user to the Amavis group root@server:~# adduser clamav amavis The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <tt>/etc/default/spamassassin</tt> to activate the Spamassassin daemon. Change <tt>ENABLED=0</tt> to: ENABLED=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <tt>/etc/amavis/conf.d/15-content_filter_mode</tt> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <tt>/etc/amavis/conf.d/20-debian-defaults</tt> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <tt>/etc/postfix/main.cf</tt> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <tt>/etc/postfix/master.cf</tt> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <tt>recipient_delimiter</tt> option on Postfix is rather neat. Given a <tt>recipient_delimiter</tt> of '<tt>.</tt>', messages to <tt>user.ext1@domain1.com</tt> are handled in an interesting way. If <tt>user.ext1@domain1.com</tt> is a valid user, the message is put in their mailbox. If <tt>user.ext1@domain1.com</tt> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <tt>user@domain1.com</tt>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <tt>/etc/postfix/main.cf</tt> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <tt>/etc/postfix/header_checks</tt> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <tt>/etc/postfix/main.cf</tt> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com maximal_queue_lifetime = 21d * Reload the configuration root@server:~# postfix reload That handles getting mail into the server, and into other servers. The next page deals with getting them out again, using [[Dovecot server setup|Dovecot as a LDA]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 58fb194ded6ec305c8fd2c44194ec462ab87e969 2177 2154 2009-12-02T11:37:18Z Neil 2 /* Configure Postfix as a backup MX server for other domains */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki.dovecot.org Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki.dovecot.org/LDA Deliver] local delivery agent, to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Postfix, TLS, and the Amavis + SpamAssassin + ClamAV are here. Dovecot, Deliver, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Basic Postfix with virtual users == Here we get Postfix running, by sending mail to other sites, and receiving mail and putting it in virtual users' mailboxes. I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /home/vmail/domain1.com/user1/ /user2/ /domain2.com/user1/ /user2/ /domain3.com/user1/ ...and so on. The <tt>vmail</tt> user will be the one used for email transport into and out of these virtual mailboxes. * Install postfix root@server:~# apt-get install postfix mailx * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update <tt>/etc/postfix/main.cf</tt> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <tt>/etc/postfix/vhosts</tt> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <tt>/etc/postfix/vmaps</tt> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <tt>/etc/postfix/vmaps</tt> is changed) * Create aliases for the users in <tt>/etc/postfix/valiases</tt> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <tt>/etc/postfix/valiases</tt> is changed) * Create <tt>/etc/aliases</tt> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <tt>telnet</tt> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <tt>mail</tt> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <tt>/etc/postfix/main.cf</tt> file to point to the certificates # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <tt>quit</tt> to end the session. SASL is mentioned in various places, but it isn't needed. SASL allows access to the SMTP server from remote sites. This Postfix installation is configured so that only machines on my LAN can use it. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <tt>/etc/mailname</tt> mydomain.com * Install the packages: root@server:~# apt-get install amavis-new spamassassin clamav === ClamAV === * Add the ClamAV user to the Amavis group root@server:~# adduser clamav amavis The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <tt>/etc/default/spamassassin</tt> to activate the Spamassassin daemon. Change <tt>ENABLED=0</tt> to: ENABLED=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <tt>/etc/amavis/conf.d/15-content_filter_mode</tt> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <tt>/etc/amavis/conf.d/20-debian-defaults</tt> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <tt>/etc/postfix/main.cf</tt> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <tt>/etc/postfix/master.cf</tt> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <tt>recipient_delimiter</tt> option on Postfix is rather neat. Given a <tt>recipient_delimiter</tt> of '<tt>.</tt>', messages to <tt>user.ext1@domain1.com</tt> are handled in an interesting way. If <tt>user.ext1@domain1.com</tt> is a valid user, the message is put in their mailbox. If <tt>user.ext1@domain1.com</tt> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <tt>user@domain1.com</tt>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <tt>/etc/postfix/main.cf</tt> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <tt>/etc/postfix/header_checks</tt> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <tt>/etc/postfix/main.cf</tt> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload That handles getting mail into the server, and into other servers. The next page deals with getting them out again, using [[Dovecot server setup|Dovecot as a LDA]]. However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] cb041a948d03fc2ea3161b5a2a61396ada407487 2178 2177 2009-12-02T11:39:03Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki.dovecot.org Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki.dovecot.org/LDA Deliver] local delivery agent, to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are here. Dovecot, Deliver, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Basic Postfix with virtual users == Here we get Postfix running, by sending mail to other sites, and receiving mail and putting it in virtual users' mailboxes. I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /home/vmail/domain1.com/user1/ /user2/ /domain2.com/user1/ /user2/ /domain3.com/user1/ ...and so on. The <tt>vmail</tt> user will be the one used for email transport into and out of these virtual mailboxes. * Install postfix root@server:~# apt-get install postfix mailx * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update <tt>/etc/postfix/main.cf</tt> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <tt>/etc/postfix/vhosts</tt> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <tt>/etc/postfix/vmaps</tt> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <tt>/etc/postfix/vmaps</tt> is changed) * Create aliases for the users in <tt>/etc/postfix/valiases</tt> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <tt>/etc/postfix/valiases</tt> is changed) * Create <tt>/etc/aliases</tt> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <tt>telnet</tt> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <tt>mail</tt> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <tt>/etc/postfix/main.cf</tt> file to point to the certificates # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <tt>quit</tt> to end the session. SASL is mentioned in various places, but it isn't needed. SASL allows access to the SMTP server from remote sites. This Postfix installation is configured so that only machines on my LAN can use it. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <tt>/etc/mailname</tt> mydomain.com * Install the packages: root@server:~# apt-get install amavis-new spamassassin clamav === ClamAV === * Add the ClamAV user to the Amavis group root@server:~# adduser clamav amavis The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <tt>/etc/default/spamassassin</tt> to activate the Spamassassin daemon. Change <tt>ENABLED=0</tt> to: ENABLED=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <tt>/etc/amavis/conf.d/15-content_filter_mode</tt> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <tt>/etc/amavis/conf.d/20-debian-defaults</tt> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <tt>/etc/postfix/main.cf</tt> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <tt>/etc/postfix/master.cf</tt> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <tt>recipient_delimiter</tt> option on Postfix is rather neat. Given a <tt>recipient_delimiter</tt> of '<tt>.</tt>', messages to <tt>user.ext1@domain1.com</tt> are handled in an interesting way. If <tt>user.ext1@domain1.com</tt> is a valid user, the message is put in their mailbox. If <tt>user.ext1@domain1.com</tt> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <tt>user@domain1.com</tt>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <tt>/etc/postfix/main.cf</tt> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <tt>/etc/postfix/header_checks</tt> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <tt>/etc/postfix/main.cf</tt> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 6bd07f1b4f30c8c00ffd1e9bd461e596b9075830 0 1541 2155 1796 2009-10-23T13:35:32Z Neil 2 /* Hostname */ Updated instructions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/resolv.conf</code> to include domain domain.tld search domain.tld * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 * Change <tt>/etc/resolv.conf</tt> to nameserver 192.168.1.1 (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; [[Category:Server setup]] bd4987f9f8a1eb082e3f11d3adeeae4af7f3a979 0 1526 2156 2128 2009-11-20T12:18:50Z Neil 2 Added section on Right Alt key wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the right Alt key == Edit <code>~/.xmodmaprc</code> to have the line keycode 113 = Alt_R ISO_Level3_Shift Multi_key Then run user@desktop:~$ xmodmap ~/.xmodmaprc and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.xmodmaprc</code> has the details. [[Category:Server setup]] 466f20f1462ee3f1e01b64626686312efef22b8b 2184 2156 2009-12-18T11:40:08Z Neil 2 /* Enable the right Alt key */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.xmodmaprc</code> to have the line keycode 113 = Mode_switch Then run user@desktop:~$ xmodmap ~/.xmodmaprc and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.xmodmaprc</code> has the details. [[Category:Server setup]] 4c91685e1ce6e1756409043b29646ccc6ec17917 2185 2184 2009-12-18T14:04:05Z Neil 2 /* Enable the Compose key and set AltGr key as Mode_swich */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.xmodmaprc</code> to have the line keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.xmodmaprc and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.xmodmaprc</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 5b895a00923209f16ae9899d203c2dfbf9d2536c 2186 2185 2010-01-11T10:12:19Z Neil 2 /* Enable the Compose key and set AltGr key as Mode_swich */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu->System->Hardware Drivers Manager and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 2affd46c6631f59f446ea984b56d20e6d1a7cd95 0 1587 2157 2062 2009-11-20T14:35:23Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. === Add another identity to a key === === Publish a public key === Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == Now you can sign and encrypt messages sent with the desktop client, the next step is to allow the same functionality with the webmail client. === Install the GPG plugin for GPG === * Download the plugin and place it in <code>/usr/share/squirrelmail/plugins/</code> * Uncompress the plugin root@server:~# cd /usr/share/squirrelmail/plugins/ root@server:/usr/share/squirrelmail/plugins# tar -xzf gpg-2.1.tar.gz * Run the configuration script root@server:~# cd /etc/squirrelmail/ root@server:/etc/squirrelmail# ./conf.pl Select option '8' (plugins); on the next screen, select the number for the GPG plugin; then select 'S' then 'Q' * Check Squirrelmail is still running: browse to <nowiki>http://your-squirrelmail-location/src/configtest.php</nowiki> === Install your GPG keypair in the server's keyring === == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] 0d8c14ec6dca1358ec10023898ebd914d7e7d99b 2159 2157 2009-11-20T14:40:17Z Neil 2 /* Allowing GnuPG signing in Squirrelmail */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. === Add another identity to a key === === Publish a public key === Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == Now you can sign and encrypt messages sent with the desktop client, the next step is to allow the same functionality with the webmail client. See [[Webmail setup]] for setting up Squirrelmail. === Install the GPG plugin for GPG === * Download the plugin and place it in <code>/usr/share/squirrelmail/plugins/</code> * Uncompress the plugin root@server:~# cd /usr/share/squirrelmail/plugins/ root@server:/usr/share/squirrelmail/plugins# tar -xzf gpg-2.1.tar.gz * Run the configuration script root@server:~# cd /etc/squirrelmail/ root@server:/etc/squirrelmail# ./conf.pl Select option '8' (plugins); on the next screen, select the number for the GPG plugin; then select 'S' then 'Q' * Check Squirrelmail is still running: browse to <nowiki>http://your-squirrelmail-location/src/configtest.php</nowiki> === Install your GPG keypair in the server's keyring === == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] a63ec5f7147a02bb11ead4d4f4c4722807cab743 2160 2159 2009-11-20T14:45:30Z Neil 2 /* GnuPG for Email */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. == Import a secret/public key pair from another source == === Add another identity to a key === === Publish a public key === Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == Now you can sign and encrypt messages sent with the desktop client, the next step is to allow the same functionality with the webmail client. See [[Webmail setup]] for setting up Squirrelmail. === Install the GPG plugin for GPG === * Download the plugin and place it in <code>/usr/share/squirrelmail/plugins/</code> * Uncompress the plugin root@server:~# cd /usr/share/squirrelmail/plugins/ root@server:/usr/share/squirrelmail/plugins# tar -xzf gpg-2.1.tar.gz * Run the configuration script root@server:~# cd /etc/squirrelmail/ root@server:/etc/squirrelmail# ./conf.pl Select option '8' (plugins); on the next screen, select the number for the GPG plugin; then select 'S' then 'Q' * Check Squirrelmail is still running: browse to <nowiki>http://your-squirrelmail-location/src/configtest.php</nowiki> === Install your GPG keypair in the server's keyring === == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] a3f87e6f2e35169113436c5fade6876547088b1e 2161 2160 2009-11-20T14:46:27Z Neil 2 /* Install the GPG plugin for GPG */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. == Import a secret/public key pair from another source == === Add another identity to a key === === Publish a public key === Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == Now you can sign and encrypt messages sent with the desktop client, the next step is to allow the same functionality with the webmail client. See [[Webmail setup]] for setting up Squirrelmail. === Install the GPG plugin for GPG === * Download [http://squirrelmail.org/plugin_view.php?id=153 the GPG plugin] from the [http://squirrelmail.org/plugins.php Squirrelmail plugins page] and place it in <code>/usr/share/squirrelmail/plugins/</code> * Uncompress the plugin root@server:~# cd /usr/share/squirrelmail/plugins/ root@server:/usr/share/squirrelmail/plugins# tar -xzf gpg-2.1.tar.gz * Run the configuration script root@server:~# cd /etc/squirrelmail/ root@server:/etc/squirrelmail# ./conf.pl Select option '8' (plugins); on the next screen, select the number for the GPG plugin; then select 'S' then 'Q' * Check Squirrelmail is still running: browse to <nowiki>http://your-squirrelmail-location/src/configtest.php</nowiki> === Install your GPG keypair in the server's keyring === == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] 9ca5c5ae489858cd4e9a62936ddc4f85de67cbe8 2162 2161 2009-11-20T15:19:44Z Neil 2 /* Allowing GnuPG signing in Squirrelmail */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. == Import a secret/public key pair from another source == === Add another identity to a key === === Publish a public key === Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == There is a plugin for this, but it hasn't been updated in a long time and no longer works. == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] 1132babc3936f8ae9efa0d273abc1a0a45a33daf 0 1535 2158 1919 2009-11-20T14:39:30Z Neil 2 Reorganised wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy. I use [http://www.squirrelmail.org/index.php SquirrelMail] as a webmail interface. To install Squirrelmail, install the packages: root@server:~# apt-get install squirrelmail ibritish (which also includes a British English dictionary). SquirrelMail has a very nice configuration script that does just about all the work for you, given that the [[Dovecot server setup|IMAP server]] and [[Web server setup|Web server]] are already working. Simply run the script: root@server:~# squirrelmail-configure select 'D' to configure using Dovecot, and follow the instructions. As SquirrelMail, Apache, and Dovecot are all running on the same machine, there's no need to use IMAPS to secure the communication between them. Next, you need to modify the <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> file to point to SquirrelMail. Adjust the <code>DocumentRoot</code> setting in the secure section of that file to DocumentRoot /usr/share/squirrelmail Restart the server: root@server:~# /etc/init.d/apache2 restart And you should have a SquirrelMail webmail site running! Check the configuration file by browsing to <nowiki>https://your-squirrelmail-location/src/configtest.php</nowiki>. == Change the login logo == If you want to change the logo on the login screen, replace the file <code>/usr/share/squirrelmail/images/sm_logo.png</code> with another image, making sure it's the same width and height as the original. The site's favicon is <code>/usr/share/squirrelmail/favicon.ico</code>. == Adding cryptography support == See the instructions for [[Cryptography setup#Allowing_GnuPG_signing_in_Squirrelmail|adding cryptography features to Squirrelmail]]. == Updating and migrating == To migrate from a previous installation, copy the files in <code>/var/lib/squirrelmail/data</code> to keep users' preferences. == See also == * [http://www.squirrelmail.org/documentation/ SquirrelMail documentation] * [https://help.ubuntu.com/community/Squirrelmail Hints for SquirrelMail on Ubuntu] [[Category:Server setup]] b3162ca1363fe43459ab23bd5e97e80ac1c063d5 2163 2158 2009-11-20T15:20:11Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy. I use [http://www.squirrelmail.org/index.php SquirrelMail] as a webmail interface. To install Squirrelmail, install the packages: root@server:~# apt-get install squirrelmail ibritish (which also includes a British English dictionary). SquirrelMail has a very nice configuration script that does just about all the work for you, given that the [[Dovecot server setup|IMAP server]] and [[Web server setup|Web server]] are already working. Simply run the script: root@server:~# squirrelmail-configure select 'D' to configure using Dovecot, and follow the instructions. As SquirrelMail, Apache, and Dovecot are all running on the same machine, there's no need to use IMAPS to secure the communication between them. Next, you need to modify the <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> file to point to SquirrelMail. Adjust the <code>DocumentRoot</code> setting in the secure section of that file to DocumentRoot /usr/share/squirrelmail Restart the server: root@server:~# /etc/init.d/apache2 restart And you should have a SquirrelMail webmail site running! Check the configuration file by browsing to <nowiki>https://your-squirrelmail-location/src/configtest.php</nowiki>. == Change the login logo == If you want to change the logo on the login screen, replace the file <code>/usr/share/squirrelmail/images/sm_logo.png</code> with another image, making sure it's the same width and height as the original. The site's favicon is <code>/usr/share/squirrelmail/favicon.ico</code>. == Updating and migrating == To migrate from a previous installation, copy the files in <code>/var/lib/squirrelmail/data</code> to keep users' preferences. == See also == * [http://www.squirrelmail.org/documentation/ SquirrelMail documentation] * [https://help.ubuntu.com/community/Squirrelmail Hints for SquirrelMail on Ubuntu] [[Category:Server setup]] 1060918f2d5d32c2f7b41876193fcad21c9c2250 2179 2163 2009-12-02T11:41:26Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I use [http://www.squirrelmail.org/index.php SquirrelMail] as a webmail interface. To install Squirrelmail, install the packages: root@server:~# apt-get install squirrelmail ibritish (which also includes a British English dictionary). SquirrelMail has a very nice configuration script that does just about all the work for you, given that the [[Dovecot server setup|IMAP server]] and [[Web server setup|Web server]] are already working. Simply run the script: root@server:~# squirrelmail-configure select 'D' to configure using Dovecot, and follow the instructions. As SquirrelMail, Apache, and Dovecot are all running on the same machine, there's no need to use IMAPS to secure the communication between them. Next, you need to modify the <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> file to point to SquirrelMail. Adjust the <code>DocumentRoot</code> setting in the secure section of that file to DocumentRoot /usr/share/squirrelmail Restart the server: root@server:~# /etc/init.d/apache2 restart And you should have a SquirrelMail webmail site running! Check the configuration file by browsing to <nowiki>https://your-squirrelmail-location/src/configtest.php</nowiki>. == Change the login logo == If you want to change the logo on the login screen, replace the file <code>/usr/share/squirrelmail/images/sm_logo.png</code> with another image, making sure it's the same width and height as the original. The site's favicon is <code>/usr/share/squirrelmail/favicon.ico</code>. == Updating and migrating == To migrate from a previous installation, copy the files in <code>/var/lib/squirrelmail/data</code> to keep users' preferences. == See also == * [http://www.squirrelmail.org/documentation/ SquirrelMail documentation] * [https://help.ubuntu.com/community/Squirrelmail Hints for SquirrelMail on Ubuntu] [[Category:Server setup]] e08760c9ac78a6e252a84a882c581fa93dc21893 0 1533 2164 1998 2009-11-30T22:13:35Z Neil 2 /* Virtual hosts */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <tt>/etc/apache2/ports.conf</tt> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <tt><nowiki>www.domain.tld</nowiki></tt> == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 8743ef643f9196a8b2f3ae5e3cf51260d3902cb0 2180 2164 2009-12-02T11:42:23Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <tt>/etc/apache2/ports.conf</tt> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <tt><nowiki>www.domain.tld</nowiki></tt> == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 3a6a5fb9be05df2bc4d80fb174d844f916601155 2181 2180 2009-12-02T11:43:47Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 3a1627fe25c1112fd1b41c5a16683daeb8351048 2182 2181 2009-12-02T11:47:39Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 8904734d9a4132b0bb9c1e320a087435164d10b0 2183 2182 2009-12-02T11:48:12Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature On Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then restart the server: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] dc4c3ccd5002f697a812fc4140a398fa32b11e5b 2187 2183 2010-02-09T08:53:38Z Neil 2 /* Virtual hosts */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature On </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature On </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] ca08ad91435d7ceb404b78f5f5ac8677e20dffc7 2188 2187 2010-02-09T08:54:13Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Virtual hosts == There will be one virtual host for each site. The files for each site will reside in a separate directory under <tt>/var/www</tt> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] bd4007a1e8c5a4c50c3aaebfe662299cc0f8009d 2189 2188 2010-03-12T09:25:47Z Neil 2 /* Virtual hosts */ Fixed NameVirtualHosts warning problem wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Name-Based Virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/apache2.conf</code> to include the two <code>NameVirtualHost</code> lines: # Include the virtual host configurations: NameVirtualHost *:80 NameVirtualHost *:443 Include /etc/apache2/sites-enabled/ There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 23e49f8dac2fccc58c1a325088e3c1a60f492ac3 2190 2189 2010-03-12T09:30:58Z Neil 2 /* Name-Based Virtual hosts */ Added how to make a site default wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/apache2.conf</code> to include the two <code>NameVirtualHost</code> lines: # Include the virtual host configurations: NameVirtualHost *:80 NameVirtualHost *:443 Include /etc/apache2/sites-enabled/ There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === If you want one virtual host to be the default site served when nothing else matches, remove the <code>default</code> site from the list of available sites: root@server:~# a2dissite /etc/apache2/sites-available/default Then, for the site you want to be the default one, modify its configuration file to start with the line: <VirtualHost _default_:80> and remove the <code>ServerName</code> line. Then reload the configuration: root@server:~# /etc/init.d/apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> NameVirtualHost *:80 <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> NameVirtualHost *:443 <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] eb700ebc57b95a8e85d1b662d2fb373b63b02b55 2191 2190 2010-03-12T09:32:19Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/apache2.conf</code> to include the two <code>NameVirtualHost</code> lines: # Include the virtual host configurations: NameVirtualHost *:80 NameVirtualHost *:443 Include /etc/apache2/sites-enabled/ There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === If you want one virtual host to be the default site served when nothing else matches, remove the <code>default</code> site from the list of available sites: root@server:~# a2dissite /etc/apache2/sites-available/default Then, for the site you want to be the default one, modify its configuration file to start with the line: <VirtualHost _default_:80> and remove the <code>ServerName</code> line. Then reload the configuration: root@server:~# /etc/init.d/apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 262660119083f3db1c4960aefc9be792ebc5a70b 0 1594 2165 2009-11-30T22:58:09Z Neil 2 New page: {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |ali... wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * www-data [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * www-data [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null === Mail servers === == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 147503dcf7cc05777455b804de76eb3cb25e9379 2166 2165 2009-11-30T23:19:49Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * www-data [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * www-data [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null === Mail servers === == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 05b2693b7cf3d8788a19be587ecca43e1e66490b 2171 2166 2009-11-30T23:22:52Z Neil 2 /* Websites */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * www-data [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * www-data [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null === Mail servers === == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] ef64d8a1e4b19bdb2babe469f2a8d0928a523216 2172 2171 2009-11-30T23:23:12Z Neil 2 /* Installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * www-data [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * www-data [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null === Mail servers === == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 96c8e36abfa7e83c27f64631bf87460daf784075 2173 2172 2009-12-01T20:15:10Z Neil 2 /* Websites */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] d9e01b4c94f4d69b1ead6d32fd73ac1c24aadb07 2174 2173 2009-12-01T20:40:21Z Neil 2 /* Mail servers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="www.domain.tld" HostAliases="localhost 127.0.0.1 www.domain.tld" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 * Add a line to the <code>/etc/cron.d/awstats</code> file, the same as the lines for the web sites: 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=mail -update >/dev/null == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] a08a00aa85b8a78da0f6cbbdff3aff6d34ad5388 2175 2174 2009-12-01T20:41:12Z Neil 2 /* Mail servers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="www.domain.tld" HostAliases="localhost 127.0.0.1 www.domain.tld" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 * Add a line to the <code>/etc/cron.d/awstats</code> file, the same as the lines for the web sites: 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=mail -update >/dev/null == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] ace545940382d59652535333be967331e07df8b9 2176 2175 2009-12-01T20:43:55Z Neil 2 /* Mail servers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 * Add a line to the <code>/etc/cron.d/awstats</code> file, the same as the lines for the web sites: 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=mail -update >/dev/null == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 2ef0e5fdc1102d413591b814febb5d261e2e8b58 0 1540 2168 2098 2009-11-30T23:20:37Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, adjust the datestamps in the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. [[Category:Server setup]] 6247f4399f23bced5114990d99bf7d6c25777b55 0 1515 2170 2149 2009-11-30T23:21:56Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 8.04, but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 620f91e568822436bfda03727bf99f41c90b6292 0 1534 2192 1792 2010-03-12T09:33:15Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/upload . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data upload root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# /etc/init.d/apache2 reload * Now got to <tt><nowiki>http://site.domain.tld/mediawiki/</nowiki></tt> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <tt>config/LocalSettings.php</tt> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <tt>$IP</tt> at the top of the file. * With the wiki set up, remove the <tt>config</tt> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <tt>LocalSettings.php</tt> file. The changes are marked below with <tt>###</tt> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <tt>/mediawiki/</tt> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <tt>/etc/apache2/sites-available/site.domain.tld</tt> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <tt><nowiki>http://site.domain.tld/Main_Page</nowiki></tt> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <tt>LocalSettings.php</tt>, modify the <tt>$wgEnableUploads</tt> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <tt>LocalSettings.php</tt> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <tt>LocalSettings.php</tt> file as shown. Add the lines just below the <tt>$wgEnableUploads</tt> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <tt>mediawiki-math</tt> pagkage is installed, it can be enabled with a simple edit of <tt>LocalSettings.php</tt>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <tt>update.php</tt> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <tt>AdminSettings.php</tt> file. Instead, do the following: * Take a backup of the wiki database * Set up the new wiki, following the instructions as above, but without deleting the <tt>config</tt> directory * Overwrite the wiki database with the backed-up version * Remove the <tt>LocalSettings.php</tt> file * Go back to the wiki's config page. This will update the database schema * Once that's done, copy the new <tt>LocalSettings.php</tt> into the Mediawiki root and continue with the configuration. == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 249edea3e2b3545fc31f0056bdd15a99a993b284 0 1579 2193 2115 2010-03-12T09:34:10Z Neil 2 wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 2e52bd801f0098de8ca1e81894f097ac7746cd92 0 1592 2194 2123 2010-03-12T09:34:45Z Neil 2 wikitext text/x-wiki I've installed the Rails task tracker [http://www.getontracks.org/ Tracks] in my [[Ruby on Rails installation|Rails server]], running Apache2, Passenger, and MySQL on Ubuntu 8.04. There were a couple of changes that I needed to do to make it all work. Most of these notes are shamelessly cribbed from the [http://tjn.me/2009/06/install-tracks-17-on-ubuntu/ TJN blog post]. == Install Rails on the server == This process is described on a separate [[Ruby on Rails installation]] page. == Create the database and user == * As per the [[Ruby on Rails deployment|normal production environment step]], create the necessary database and user. root@server:~# mysql -u root -p mysql> create database tracks; mysql> grant all on tracks.* to 'tracksuser'@'localhost' identified by 'SecretPassword'; mysql> quit; * We can also create a user that Tracks runs as root@server:~# adduser tracks == Install the files == * Download Tracks from the [http://www.getontracks.org/ Tracks homepage] and copy the file into <code>/var/www/tracks.domain.tld</code>. * Unzip the files root@server:~# cd /var/www/tracks.domain.tld root@server:/var/www/tracks.domain.tld# unzip tracks-1.7.zip root@server:/var/www/tracks.domain.tld# cd tracks-1.7 * Now, fix the ownership and permissions of the files root@server:/var/www/tracks.domain.tld/tracks-1.7# chown -R tracks:tracks * root@server:/var/www/tracks.domain.tld/tracks-1.7# chmod -R 664 * root@server:/var/www/tracks.domain.tld/tracks-1.7# find -type d -exec chmod a+x '{}' \; root@server:/var/www/tracks.domain.tld/tracks-1.7# chmod -R a+x script/* == Update the Tracks configuration == * Edit <code>config/database.yml</code> to use the MySQL database: production: adapter: mysql database: tracks host: localhost username: tracksuser password: SecretPassword * Edit <code>config/site.yml</code> to change the password salt: # This is the 'salt' to add to the password before it is encrypted # You need to change this to something unique for yourself # salt: "change-me" salt: "SecretSalt" :(substitute your own salt, of course) == Populate the database == root@server:/var/www/tracks.domain.tld/tracks-1.7# rake db:migrate RAILS_ENV=production == Allow recurrent todos == Modify line 76 of <code>app/views/recurring_todos/_recurring_todo_form.erb</code> to be this: text_field(:recurring_todo, :start_from, "value" => format_date(Time.now), "size" => 12, "class" => "Date", "onfocus" => "Calendar.setup", "tabindex" => 6, "autocomplete" => "off") %><br/> :(instead of a reference to <code>format_date(Time.today)</code> ) == Create the Apache virtual host == * Create the virtual host file, <code>/etc/apache2/sites-available/tracks.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/tracks.domain.tld/tracks-1.7/public ServerName tracks.domain.tld RewriteEngine On RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Remove Tracks's <code>.htaccess</code> file: root@server:var/www/tracks.domain.tld/tracks-1.7# mv public/.htaccess public/dontneedhtaccess * Enable the site: root@server:~# a2ensite tracks.domain.tld root@server:~# /etc/init.d/apache2 reload == Allow restarts of Passenger == * Create the <code>tmp</code> directory: root@server:~# mkdir -p /var/www/tracks.domain.tld/tracks-1.7/tmp * To restart Passenger, issue this command: root@server:~# touch /var/www/tracks.domain.tld/tracks-1.7/tmp/restart.txt [[Category:Rails]] [[Category:Ruby]] d3605fcb2573977e230e276770207977b1b600e0 0 1 2195 2109 2010-03-31T21:53:14Z Neil 2 /* Neil */ Added juggling wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. I also have a few [[downloadable programs]] (nothing exciting). And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] has lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. c82e66df78f57d3b34970c6c8f0b0d1c87abf3ed 2196 2195 2010-03-31T21:54:08Z Neil 2 /* Jenni */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. I also have a few [[downloadable programs]] (nothing exciting). And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] has lots of animations of tricks I can't do. == Jenni == No content currently. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 90ec06521a5fd722ec6054972120422185389f46 2197 2196 2010-04-02T10:08:53Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. I also have a few [[downloadable programs]] (nothing exciting). And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == No content currently. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. c456a6622529e7d69af1a2ebee7ad6b75310c00c 2198 2197 2010-04-23T15:24:27Z Neil 2 /* Jenni */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. I also have a few [[downloadable programs]] (nothing exciting). And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 497f22d7f5d4329a4581a1c7934add1f9e888b8b 0 1543 2199 2135 2010-05-12T09:34:26Z Neil 2 /* Add a bespoke iptables firewall */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] a792345eb11a19b96da58436d4bdfb9894f50209 2200 2199 2010-05-12T09:34:54Z Neil 2 /* Iptables.rules */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 2387737b9029c1a4b9ba568184a4c3a27c1370f2 0 1544 2201 2085 2010-05-12T11:11:50Z Neil 2 /* Chrooting the nameservers */ Updated for Ubuntu 10.04 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld NS desktop.domain.tld ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/named root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run root@server:~# chown -R bind:bind /chroot/bind/var/cache * Modify the file <code>/etc/default/bind9</code> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * '''For Ubuntu 8.04:''' Modify <code>/etc/default/syslogd</code> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * '''For Ubuntu 10.04:''' Create <code>/etc/rsyslog.d/bind-chroot.conf</code> to log messages from the chrooted BIND: $AddUnixListenSocket /chroot/bind/dev/log * Make AppArmor allow BIND access to the chroot jail. Modify <code>/etc/apparmor.d/usr.sbin.named</code> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/lib/bind/ rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/cache/bind/ rw, /chroot/bind/var/run/named/named.pid w, /chroot/bind/var/run/named/session.key w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, system logger, and BIND: root@server:~# /etc/init.d/apparmor restart ** '''For Ubuntu 8.04:''' root@server:~# /etc/init.d/sysklogd restart ** '''For Ubuntu 10.04:''' root@server:~# service rsyslog restart root@server:~# /etc/init.d/bind9 restart * Check <code>/var/log/syslog</code> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 1dadda6b5526064794ad0888a55c93ee8ae46acb 0 1545 2202 2032 2010-05-12T11:18:56Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server mysql-client == Passwords == The first thing is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <tt>hostname</tt> is the name of this host, e.g. <tt>server.domain.tld</tt>. * Add passwords to the <tt>root</tt> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); == Remote access == If you want to access MySQL databases from a remote machine, you'll need to have MySQL accept connections across the network. Edit <code>/etc/mysql/my.cnf</code> and change the <code>bind-address</code> entry to be the IP of the machine: bind-address = 192.168.2.252 The restart MySQL: root@server:~# /etc/init.d/mysql restart Note that the [[Firewall setup|firewall]] is configured to only allow connections to MySQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] 6db58d88c8210ce8b3d3c242a04dd703c0333388 2203 2202 2010-05-12T12:24:57Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server mysql-client == Passwords == The first thing is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <code>host_name</code> is the name of this host, e.g. <code>server</code>. :(seemingly not needed for Ubuntu 10.04) * Add passwords to the <code>root</code> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'127.0.0.1' = PASSWORD('newpwd'); == Remote access == If you want to access MySQL databases from a remote machine, you'll need to have MySQL accept connections across the network. Edit <code>/etc/mysql/my.cnf</code> and change the <code>bind-address</code> entry to be the IP of the machine: bind-address = 192.168.2.252 The restart MySQL: root@server:~# /etc/init.d/mysql restart Note that the [[Firewall setup|firewall]] is configured to only allow connections to MySQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] e0b0a14838ff62b0f361071fd4d407b4c32a43d3 0 1533 2204 2191 2010-05-12T12:32:36Z Neil 2 /* Getting Apache2 running */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/apache2.conf</code> to include the two <code>NameVirtualHost</code> lines: # Include the virtual host configurations: NameVirtualHost *:80 NameVirtualHost *:443 Include /etc/apache2/sites-enabled/ There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === If you want one virtual host to be the default site served when nothing else matches, remove the <code>default</code> site from the list of available sites: root@server:~# a2dissite /etc/apache2/sites-available/default Then, for the site you want to be the default one, modify its configuration file to start with the line: <VirtualHost _default_:80> and remove the <code>ServerName</code> line. Then reload the configuration: root@server:~# /etc/init.d/apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] a9fa910ce87fce3e47021d5c1dbb4f801cb532cd 0 1526 2205 2186 2010-05-13T09:52:33Z Neil 2 /* Install proprietary graphics card driver */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Install proprietary graphics card driver == I have a NVidia GeForce 6600 video card. The proprietary video card driver uses the card's GPU for better graphics. To check that the card is supported, do root@desktop:~# lspci | grep nvidia and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu -> System -> Hardware Drivers Manager (in Kubuntu 10.04, it's K -> Applications -> System -> Hardware Drivers) and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 2cfbef807f820f1f058a21f559d5ec50d1f4bb44 2207 2205 2010-05-13T15:02:41Z Neil 2 /* Install proprietary graphics card driver */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] cc2719c8a4da80e27968bc2597b12c4895ac19c9 2208 2207 2010-05-13T15:03:26Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] a41dff39f53395d28923f0a2d322ba73e428af03 2209 2208 2010-05-13T21:21:14Z Neil 2 /* Emacs configuration */ Added Akonadi section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. Unfortunately, it doesn't work very well. To get it working, stop it, remove the existing MySQL database it uses, use a better config file, and restart it. user@desktop:~$ akonadictl stop user@desktop:~$ rm -r ~/.local/share/akonadi/db_data/* user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf user@desktop:~$ akonadictl start Then, Kontact should start. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] aed31dfcb4d824d0e68a41639a855f02fa4d8df1 2210 2209 2010-05-14T14:23:12Z Neil 2 /* Get Akonadi (and Kontact) to start */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * [http://eco.netvibes.com/tools/firefox AddToNetvibes] == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server user@desktop:~$ sudo apt-get install mysql-server-5.1 user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start Ignore any error messages emitted by MySQL during its steps. Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on Fedora boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 377a4f970137fd635c659a141c447054e5e58594 2211 2210 2010-05-16T08:52:45Z Neil 2 /* Firefox extensions */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server user@desktop:~$ sudo apt-get install mysql-server-5.1 user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start Ignore any error messages emitted by MySQL during its steps. Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on Fedora boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] dfa262e01d952c79bd8a80331ac9bf2ddc04458c 2212 2211 2010-05-17T09:58:53Z Neil 2 /* Get Akonadi (and Kontact) to start */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server user@desktop:~$ sudo apt-get install mysql-server-5.1 user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start Ignore any error messages emitted by MySQL during its steps. Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] f98c9efef12eea251915462e586faf4f9192a912 2213 2212 2010-05-17T10:05:45Z Neil 2 /* Get Akonadi (and Kontact) to start */ Added KMail domain section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server user@desktop:~$ sudo apt-get install mysql-server-5.1 user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start Ignore any error messages emitted by MySQL during its steps. Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 479790b2d0b5f05239d67fa331a297a4682a1a69 2214 2213 2010-05-19T10:46:45Z Neil 2 /* Get Akonadi (and Kontact) to start */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] c6dd1549f45d992ceb5c224d1d232f71531e2ecb 2215 2214 2010-05-19T11:10:44Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 9207e461a0f01fbf721a536039e42a098f4b336d 2216 2215 2010-05-21T14:09:19Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 37716ca75290bd08478d92d01b79dd580a5e2666 2217 2216 2010-05-24T13:17:18Z Neil 2 /* Install additional packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 6733a032d9f5309744f2f3195a185e8a22287c56 2218 2217 2010-05-25T17:44:56Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 56e6ef941e76bd46b638370bc461f8e3b3282a08 2223 2218 2010-06-02T09:19:56Z Neil 2 /* Install and use Sun Java */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. <code>~/.Xmodmap</code> has the details. The right Windows key now acts as the Compose key, allowing you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] a4541c58b34ada4d00f986f38e98e1cabd6eee62 2229 2223 2010-07-17T20:03:04Z Neil 2 /* Enable the Compose key and set AltGr key as Mode_swich */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Mode switch key, and the 'right-click menu' key as the Compose key. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch # AltGr keycode 116 = Multi_key # Right-click menu: Compose Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. For instance AltGr+5 will generate ½. <code>~/.Xmodmap</code> has the details. The Compose key allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 5239fb45b9bc365db8de20c052dc345c34a6c522 2230 2229 2010-07-17T20:04:02Z Neil 2 /* Enable the Compose key and set AltGr key as Mode_swich */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Mode switch key, and the 'right-click menu' key as the Compose key. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. For instance AltGr+5 will generate ½. <code>~/.Xmodmap</code> has the details. The Compose key allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] d79664cda262a0a72c5897b5f8d74228019e22ca 0 1516 2206 1862 2010-05-13T15:02:33Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Router setup]] |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/products/WhatIsUbuntu/serveredition Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a couple of GB each for <code>/var</code> and <code>/tmp</code>, and everything else just goes into one large partition. The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <code>build-essentials</code> (for basic software development) and <code>openssh-server</code> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. == Install proprietary graphics card driver == I have a NVidia GeForce G210 video card. The proprietary video card driver uses the card's GPU for better graphics; the open source one makes KDE hang all too often. This means you need to install the proprietary driver as soon as the OS is installed on the desktop machine. To check that the card is supported, do: root@desktop:~# lspci | grep -i nvidia 02:00.0 VGA compatible controller: nVidia Corporation GT218 [GeForce G210] (rev a2) note the code at the start of the line, and use that to find the PCI ID of the card: root@desktop:~# lspci -n | grep 02:00 02:00.0 0300: 10de:0a60 (rev a2) and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:0a60</code>; the GeForce 6600 on the old machine was <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu -> System -> Hardware Drivers Manager (in Kubuntu 10.04, it's K -> Applications -> System -> Hardware Drivers) and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. [[Category:Server setup]] 28d23f977a23e1c0d3809ee75e6b8111be6cfc6f 0 1563 2219 1918 2010-05-25T20:28:38Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} == Install HPLIP == My printer (an HP 1312) is supported by HPLIP, but is only supported for scanning with a plugin. Unfortunately, the HPLIP that comes with Ubuntu breaks when it tries to install a plugin. This is fixed with the installation of HPLIP direct from HP. * Remove the existing packages root@desktop:~# apt-get purge hplip hplip-gui hpijs hplip-cups libhpmud0 hpijs-ppds foomatic-db-hpijs * Download [http://hplipopensource.com/hplip-web/downloads.html HPLIP] from HP. * Run <code>hplip-3.10.5.run</code> to install it. Don't run it as root, but as an ordinary user. The [http://hplipopensource.com/hplip-web/install/install/index.html instructions from HP] are useful. user@desktop:~$ sh hplip-3.10.5.run * When it asks you for root access to set up the printer, cancel. * Run <code>hp-setup</code> to set up the printer with root privileges. Ignore the errors that come when it does so. root@desktop:~# sudo hp-setup (See also the [[CUPS server setup|printer setup]] page) == Server setup == (We're using the desktop machine as the server for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# apt-get xsane xsane-doc sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [http://ubuntuforums.org/showthread.php?p=4916614 this forum post] and [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# /etc/init.d/xinetd restart * Open the right port in the [[Firewall setup|firewall]] by adding these lines to <code>/etc/iptables.rules</code>. Make sure the scanner is only available to machines on this LAN. ## Network scanning (port 6566) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport saned -j ACCEPT :and restart the firewall root@desktop:~# /etc/init.d/iptables restart * As this is a multi-function printer, restart CUPS and check that it can still print to the printer. == Linux client setup == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# apt-get sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup == Use [http://sanetwain.ozuzo.net/ SaneTwain] and follow the instructions. Unfortunately, I couldn't get the scanner working reliably across a network at all, so SaneTwain didn't have a hope of working. == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] fc9d2de22bdf46c83df5d3164285e554b0b83ea9 0 1540 2220 2168 2010-05-26T09:15:24Z Neil 2 /* Anacron timing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <tt>/etc/cron.d/anacron</tt> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, use <code>touch</code> to adjust the datestamps in the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] bf383bdb9579c91618e95e683d3c61c48e087ce1 2221 2220 2010-05-27T07:10:38Z Neil 2 /* Anacron timing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 526c571d0160ff07b4735f1a60ccf5c81f8abdfa 2222 2221 2010-05-27T07:12:52Z Neil 2 /* Server backup */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Anacron timing == I found that the default timing for kicking off Anacron on the desktop machine (at around 7.30) meant that the backups were still being created when I started work on it. That was easily fixed by altering <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 7e324106e88dacf65f30dd0d1a162d57685e0193 0 1577 2224 2153 2010-06-02T16:06:26Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient-dev libruby-extras \ libyaml-ruby libzlib-ruby libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient-dev \ libyaml-ruby libzlib-ruby libruby-extras libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from source, to avoid it conflicting with <tt>apt</tt> root@server:~# wget <nowiki>http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz</nowiki> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <tt>sudo ruby rubygemsx.x.x/setup.rb</tt>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both. * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install sqlite3-ruby root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install sqlite3-ruby root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 83e08881cfbb2fa8a6d6dc1c494d28aa6f9b0591 2225 2224 2010-06-02T16:09:52Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient-dev libruby-extras \ libyaml-ruby libzlib-ruby libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@server:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient-dev \ libyaml-ruby libzlib-ruby libruby-extras libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * Build <code>gem</code> from [http://rubygems.org/pages/download source], to avoid it conflicting with <code>apt</code> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <code>sudo ruby rubygemsx.x.x/setup.rb</code>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :(The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both.) * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install sqlite3-ruby root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install sqlite3-ruby root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] c8c2c21d577ff4bad67d0c6fc4f5c37a2ce34a43 0 1531 2226 2178 2010-06-23T22:45:13Z Neil 2 /* Adding TLS */ Added SASL section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki.dovecot.org Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki.dovecot.org/LDA Deliver] local delivery agent, to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are here. Dovecot, Deliver, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Basic Postfix with virtual users == Here we get Postfix running, by sending mail to other sites, and receiving mail and putting it in virtual users' mailboxes. I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /home/vmail/domain1.com/user1/ /user2/ /domain2.com/user1/ /user2/ /domain3.com/user1/ ...and so on. The <tt>vmail</tt> user will be the one used for email transport into and out of these virtual mailboxes. * Install postfix root@server:~# apt-get install postfix mailx * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update <tt>/etc/postfix/main.cf</tt> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <tt>/etc/postfix/vhosts</tt> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <tt>/etc/postfix/vmaps</tt> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <tt>/etc/postfix/vmaps</tt> is changed) * Create aliases for the users in <tt>/etc/postfix/valiases</tt> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <tt>/etc/postfix/valiases</tt> is changed) * Create <tt>/etc/aliases</tt> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <tt>telnet</tt> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <tt>mail</tt> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <tt>/etc/mailname</tt> mydomain.com * Install the packages: root@server:~# apt-get install amavis-new spamassassin clamav === ClamAV === * Add the ClamAV user to the Amavis group root@server:~# adduser clamav amavis The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <tt>/etc/default/spamassassin</tt> to activate the Spamassassin daemon. Change <tt>ENABLED=0</tt> to: ENABLED=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <tt>/etc/amavis/conf.d/15-content_filter_mode</tt> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <tt>/etc/amavis/conf.d/20-debian-defaults</tt> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <tt>/etc/postfix/main.cf</tt> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <tt>/etc/postfix/master.cf</tt> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <tt>recipient_delimiter</tt> option on Postfix is rather neat. Given a <tt>recipient_delimiter</tt> of '<tt>.</tt>', messages to <tt>user.ext1@domain1.com</tt> are handled in an interesting way. If <tt>user.ext1@domain1.com</tt> is a valid user, the message is put in their mailbox. If <tt>user.ext1@domain1.com</tt> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <tt>user@domain1.com</tt>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <tt>/etc/postfix/main.cf</tt> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <tt>/etc/postfix/header_checks</tt> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <tt>/etc/postfix/main.cf</tt> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 96a247f4895c10a7114dd8a7e0886e03fff3afe7 0 1532 2227 1777 2010-06-23T22:57:41Z Neil 2 Added IMAPS notes wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This is all about the user-facing side of setting up a mail server. For information on setting up the server-facing side, see the [[Postfix server setup]] page. Also see the [[Webmail setup]] page. The basic source for setting up Postfix and Dovecot is the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois :(<tt>whois</tt> is needed in a moment for creating passwords) * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <tt>/etc/dovecot/dovecot.conf</tt> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = cmusieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <tt>/etc/postfix/main.cf</tt> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <tt>/etc/postfix/master.cf</tt> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Create an an empty log file for <tt>deliver</tt> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <tt>/home/vmail/domain1.com/user1/.dovecot.sieve</tt> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <tt>.dovecot.sieve</tt> file. The first time Dovecot delivers mail to this user, it will create the <tt>.dovecot.sievec</tt> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 0da2f0d6bb9509c4a44182b02ae724f7d505bdfd 2228 2227 2010-06-23T23:05:26Z Neil 2 /* Create virtual users */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This is all about the user-facing side of setting up a mail server. For information on setting up the server-facing side, see the [[Postfix server setup]] page. Also see the [[Webmail setup]] page. The basic source for setting up Postfix and Dovecot is the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois :(<tt>whois</tt> is needed in a moment for creating passwords) * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password :(If you want to change a user's password, delete their line from <code>/etc/dovecot/passwd</code> before running the <code>mkdovecotpasswd</code> script again.) * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <tt>/etc/dovecot/dovecot.conf</tt> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = cmusieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <tt>/etc/postfix/main.cf</tt> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <tt>/etc/postfix/master.cf</tt> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Create an an empty log file for <tt>deliver</tt> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <tt>/home/vmail/domain1.com/user1/.dovecot.sieve</tt> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <tt>.dovecot.sieve</tt> file. The first time Dovecot delivers mail to this user, it will create the <tt>.dovecot.sievec</tt> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] b06dd83c5e3d2865f7b2f79713d03185d2dcd583 0 1596 2231 2010-07-27T08:29:27Z Neil 2 New page: This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a sh... wikitext text/x-wiki This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. [[Category:RPGs]] [[Category:Sivavile]] d7d6b7e4faca81dfe91b5679fb8f04a3d3241bee 2232 2231 2010-07-27T08:49:30Z Neil 2 wikitext text/x-wiki This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} [[Category:RPGs]] [[Category:Sivavile]] 29ca17009d232642f3076c6cb31d1595b955813f 2233 2232 2010-07-27T09:43:50Z Neil 2 wikitext text/x-wiki =The town= This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. =Names= {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} =Character types= ==Guardians== One of the soldiers/police from Siyavile Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, 1 person. Ability: ==Trader== ==Explorer== ==Priest(ess)== ==Imperial soldier== ==Refugee== ==Prince(ess)== =Lists= ==Arms== ==Gear== ==Maps== ==Spells== ==People== ==Default== [[Category:RPGs]] [[Category:Sivavile]] c58aa8775d7944eae96a8a929efaf37c72945581 2234 2233 2010-07-27T18:48:23Z Neil 2 /* Character types */ wikitext text/x-wiki =The town= This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. =Names= {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} =Character types= ==Guardians== One of the soldiers/police from Siyavile. Don't choose this if you have low strength. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Hammerhands ==Trader (and explorer)== A traveller who sells fish and buys other goods, often in far-off places of the world. Don't choose this if you have low command or low perception. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Social training (or Danger Sense) ==Priest(ess)== You are a scholar of magic and wisdom. Don't choose this if you have both a low command and a low perception. * Choose 3 maps, 3 spells, 1 arms or gear, 1 from any list, and 1 person. * Ability: Initiation ==Imperial soldier== You have been trained by the Empire in how to fight. Don't choose this if you have low strength or low skill. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Parrying (sword skill) ==Refugee== You have fled the Empire and now seek refuge in a strange land. Don't choose this if you have low Endurance. * Choose 3 arms, 3 maps, 1 gear or spells, 1 from any list, and 1 person. * Ability: Tireless ==Prince(ess)== Your father is the king. Don't choose this if you have both low command and low strength. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Tactician. =Lists= ==Arms== ==Gear== ==Maps== ==Spells== ==People== ==Default== [[Category:RPGs]] [[Category:Sivavile]] 8c04dd97152bdb19e20bee67b67dc773f449a72b 2235 2234 2010-07-27T19:01:51Z Neil 2 /* Lists */ wikitext text/x-wiki =The town= This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. =Names= {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} =Character types= ==Guardians== One of the soldiers/police from Siyavile. Don't choose this if you have low strength. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Hammerhands ==Trader (and explorer)== A traveller who sells fish and buys other goods, often in far-off places of the world. Don't choose this if you have low command or low perception. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Social training (or Danger Sense) ==Priest(ess)== You are a scholar of magic and wisdom. Don't choose this if you have both a low command and a low perception. * Choose 3 maps, 3 spells, 1 arms or gear, 1 from any list, and 1 person. * Ability: Initiation ==Imperial soldier== You have been trained by the Empire in how to fight. Don't choose this if you have low strength or low skill. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Parrying (sword skill) ==Refugee== You have fled the Empire and now seek refuge in a strange land. Don't choose this if you have low Endurance. * Choose 3 arms, 3 maps, 1 gear or spells, 1 from any list, and 1 person. * Ability: Tireless ==Prince(ess)== Your father is the king. Don't choose this if you have both low command and low strength. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Tactician. =Lists= ==Arms== * Breastplate and helmet (2 blue) * Broadsword (2 red) * Chainmail (2 blue) * Handaxe (2 red) * Longbow (3 red, 2-handed, bulky) * Net (1 red and 1 blue) * Shield (2 blue) * Shortbow (2 red) * Shortsword (1 red) * Sling (1 red, 2-handed) * Spear (2 red) * Staff (1 red and 1 blue) * Throwing knives (1 red) * Trident (2 red) ==Gear== * Boat * Climbing gear (rope, hook, spikes) * Dagger (1 red) * Fine clothing * Healer's kit (bandages, liquor, salves) * Jewellery * Lockpicks * Money * Oilskins (1 blue) * Provisions ==Maps== * Architectural drawings * Childhood memory * Familiar spirit * First-hand account * Legends, lore, and myth * Magnetic compass * Spellbook * Star charts * Trade and travel map * Treasure map ==Spells== * A guiding shadow * A warning vision * Arrows of flame * Bath of healing light * Lightnings at command * Luck for warriors * Passage unnoticed * Sensitivity to magic * Shelter of vines * Strength of ten ==People== * Healer * Priestess * Thug * Guardian * Elder * Boatmaker * Weaponsmith * Trader * Crafter * Imperial visitor * Refugee ==Default== Otherwise, everyone gets common clothing, including hat and clothes (gear), lunch and trail snacks (gear), flint, steel, and tinder, torches if called for (gear), and local lore (map). [[Category:RPGs]] [[Category:Sivavile]] 0161a6548b2f2c50ef250be0700e1be2b0196ff5 2236 2235 2010-07-31T16:04:57Z Neil 2 wikitext text/x-wiki =The town= This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. =Names= {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} =Character types= ==Guardians== One of the soldiers/police from Siyavile. Don't choose this if you have low strength. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Hammerhands ==Trader (and explorer)== A traveller who sells fish and buys other goods, often in far-off places of the world. Don't choose this if you have low command or low perception. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Social training (or Danger Sense) ==Priest(ess)== You are a scholar of magic and wisdom. Don't choose this if you have both a low command and a low perception. * Choose 3 maps, 3 spells, 1 arms or gear, 1 from any list, and 1 person. * Ability: Initiation ==Imperial soldier== You have been trained by the Empire in how to fight. Don't choose this if you have low strength or low skill. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Parrying (sword skill) ==Refugee== You have fled the Empire and now seek refuge in a strange land. Don't choose this if you have low Endurance. * Choose 3 arms, 3 maps, 1 gear or spells, 1 from any list, and 1 person. * Ability: Tireless ==Prince(ess)== Your father is the king. Don't choose this if you have both low command and low strength. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Tactician. =Lists= ==Arms== * Breastplate and helmet (2 blue) * Broadsword (2 red) * Chainmail (2 blue) * Handaxe (2 red) * Longbow (3 red, 2-handed, bulky) * Net (1 red and 1 blue) * Shield (2 blue) * Shortbow (2 red) * Shortsword (1 red) * Sling (1 red, 2-handed) * Spear (2 red) * Staff (1 red and 1 blue) * Throwing knives (1 red) * Trident (2 red) ==Gear== * Boat * Climbing gear (rope, hook, spikes) * Dagger (1 red) * Fine clothing * Healer's kit (bandages, liquor, salves) * Jewellery * Lockpicks * Money * Oilskins (1 blue) * Provisions ==Maps== * Architectural drawings * Childhood memory * Familiar spirit * First-hand account * Legends, lore, and myth * Magnetic compass * Spellbook * Star charts * Trade and travel map * Treasure map ==Spells== * A guiding shadow * A warning vision * Arrows of flame * Bath of healing light * Lightnings at command * Luck for warriors * Passage unnoticed * Sensitivity to magic * Shelter of vines * Strength of ten ==People== * Healer * Priestess * Thug * Guardian * Elder * Boatmaker * Weaponsmith * Trader * Crafter * Imperial visitor * Refugee ==Default== Otherwise, everyone gets common clothing, including hat and clothes (gear), lunch and trail snacks (gear), flint, steel, and tinder, torches if called for (gear), and local lore (map). [[Category:RPGs]] [[Category:Siyavile]] 66c413d29c7ed59c71140e4ecc5f2536b7061f08 2241 2236 2010-08-02T10:08:58Z Neil 2 /* The town */ Added maps wikitext text/x-wiki =The town= [[Image:Siyavile-large-area.png|right|thumb|200px|The area around Siyavile]] This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. [[Image:Siyavile-local.png|right|thumb|200px|The town of Siyavile]] Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. =Names= {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} =Character types= ==Guardians== One of the soldiers/police from Siyavile. Don't choose this if you have low strength. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Hammerhands ==Trader (and explorer)== A traveller who sells fish and buys other goods, often in far-off places of the world. Don't choose this if you have low command or low perception. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Social training (or Danger Sense) ==Priest(ess)== You are a scholar of magic and wisdom. Don't choose this if you have both a low command and a low perception. * Choose 3 maps, 3 spells, 1 arms or gear, 1 from any list, and 1 person. * Ability: Initiation ==Imperial soldier== You have been trained by the Empire in how to fight. Don't choose this if you have low strength or low skill. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Parrying (sword skill) ==Refugee== You have fled the Empire and now seek refuge in a strange land. Don't choose this if you have low Endurance. * Choose 3 arms, 3 maps, 1 gear or spells, 1 from any list, and 1 person. * Ability: Tireless ==Prince(ess)== Your father is the king. Don't choose this if you have both low command and low strength. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Tactician. =Lists= ==Arms== * Breastplate and helmet (2 blue) * Broadsword (2 red) * Chainmail (2 blue) * Handaxe (2 red) * Longbow (3 red, 2-handed, bulky) * Net (1 red and 1 blue) * Shield (2 blue) * Shortbow (2 red) * Shortsword (1 red) * Sling (1 red, 2-handed) * Spear (2 red) * Staff (1 red and 1 blue) * Throwing knives (1 red) * Trident (2 red) ==Gear== * Boat * Climbing gear (rope, hook, spikes) * Dagger (1 red) * Fine clothing * Healer's kit (bandages, liquor, salves) * Jewellery * Lockpicks * Money * Oilskins (1 blue) * Provisions ==Maps== * Architectural drawings * Childhood memory * Familiar spirit * First-hand account * Legends, lore, and myth * Magnetic compass * Spellbook * Star charts * Trade and travel map * Treasure map ==Spells== * A guiding shadow * A warning vision * Arrows of flame * Bath of healing light * Lightnings at command * Luck for warriors * Passage unnoticed * Sensitivity to magic * Shelter of vines * Strength of ten ==People== * Healer * Priestess * Thug * Guardian * Elder * Boatmaker * Weaponsmith * Trader * Crafter * Imperial visitor * Refugee ==Default== Otherwise, everyone gets common clothing, including hat and clothes (gear), lunch and trail snacks (gear), flint, steel, and tinder, torches if called for (gear), and local lore (map). [[Category:RPGs]] [[Category:Siyavile]] 36544ff7e7316fd6fba3115408ae1dff4d35751b 2242 2241 2010-08-02T10:10:26Z Neil 2 wikitext text/x-wiki This is a description of the town of Siyavile, the base of our [http://www.lumpley.com/storming/ Storming the Wizard's Tower] game. =The town= [[Image:Siyavile-large-area.png|right|thumb|200px|The area around Siyavile]] This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. [[Image:Siyavile-local.png|right|thumb|200px|The town of Siyavile]] Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. =Names= {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} =Character types= ==Guardians== One of the soldiers/police from Siyavile. Don't choose this if you have low strength. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Hammerhands ==Trader (and explorer)== A traveller who sells fish and buys other goods, often in far-off places of the world. Don't choose this if you have low command or low perception. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Social training (or Danger Sense) ==Priest(ess)== You are a scholar of magic and wisdom. Don't choose this if you have both a low command and a low perception. * Choose 3 maps, 3 spells, 1 arms or gear, 1 from any list, and 1 person. * Ability: Initiation ==Imperial soldier== You have been trained by the Empire in how to fight. Don't choose this if you have low strength or low skill. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Parrying (sword skill) ==Refugee== You have fled the Empire and now seek refuge in a strange land. Don't choose this if you have low Endurance. * Choose 3 arms, 3 maps, 1 gear or spells, 1 from any list, and 1 person. * Ability: Tireless ==Prince(ess)== Your father is the king. Don't choose this if you have both low command and low strength. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Tactician. =Lists= ==Arms== * Breastplate and helmet (2 blue) * Broadsword (2 red) * Chainmail (2 blue) * Handaxe (2 red) * Longbow (3 red, 2-handed, bulky) * Net (1 red and 1 blue) * Shield (2 blue) * Shortbow (2 red) * Shortsword (1 red) * Sling (1 red, 2-handed) * Spear (2 red) * Staff (1 red and 1 blue) * Throwing knives (1 red) * Trident (2 red) ==Gear== * Boat * Climbing gear (rope, hook, spikes) * Dagger (1 red) * Fine clothing * Healer's kit (bandages, liquor, salves) * Jewellery * Lockpicks * Money * Oilskins (1 blue) * Provisions ==Maps== * Architectural drawings * Childhood memory * Familiar spirit * First-hand account * Legends, lore, and myth * Magnetic compass * Spellbook * Star charts * Trade and travel map * Treasure map ==Spells== * A guiding shadow * A warning vision * Arrows of flame * Bath of healing light * Lightnings at command * Luck for warriors * Passage unnoticed * Sensitivity to magic * Shelter of vines * Strength of ten ==People== * Healer * Priestess * Thug * Guardian * Elder * Boatmaker * Weaponsmith * Trader * Crafter * Imperial visitor * Refugee ==Default== Otherwise, everyone gets common clothing, including hat and clothes (gear), lunch and trail snacks (gear), flint, steel, and tinder, torches if called for (gear), and local lore (map). [[Category:RPGs]] [[Category:Siyavile]] 8ecf202bcd0bc1c504fcaf8ba24ef57ecbec9f48 2244 2242 2010-08-02T14:26:49Z Neil 2 [[Siyavile]] moved to [[Siyavile the town]] wikitext text/x-wiki This is a description of the town of Siyavile, the base of our [http://www.lumpley.com/storming/ Storming the Wizard's Tower] game. =The town= [[Image:Siyavile-large-area.png|right|thumb|200px|The area around Siyavile]] This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. [[Image:Siyavile-local.png|right|thumb|200px|The town of Siyavile]] Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. =Names= {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} =Character types= ==Guardians== One of the soldiers/police from Siyavile. Don't choose this if you have low strength. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Hammerhands ==Trader (and explorer)== A traveller who sells fish and buys other goods, often in far-off places of the world. Don't choose this if you have low command or low perception. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Social training (or Danger Sense) ==Priest(ess)== You are a scholar of magic and wisdom. Don't choose this if you have both a low command and a low perception. * Choose 3 maps, 3 spells, 1 arms or gear, 1 from any list, and 1 person. * Ability: Initiation ==Imperial soldier== You have been trained by the Empire in how to fight. Don't choose this if you have low strength or low skill. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Parrying (sword skill) ==Refugee== You have fled the Empire and now seek refuge in a strange land. Don't choose this if you have low Endurance. * Choose 3 arms, 3 maps, 1 gear or spells, 1 from any list, and 1 person. * Ability: Tireless ==Prince(ess)== Your father is the king. Don't choose this if you have both low command and low strength. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Tactician. =Lists= ==Arms== * Breastplate and helmet (2 blue) * Broadsword (2 red) * Chainmail (2 blue) * Handaxe (2 red) * Longbow (3 red, 2-handed, bulky) * Net (1 red and 1 blue) * Shield (2 blue) * Shortbow (2 red) * Shortsword (1 red) * Sling (1 red, 2-handed) * Spear (2 red) * Staff (1 red and 1 blue) * Throwing knives (1 red) * Trident (2 red) ==Gear== * Boat * Climbing gear (rope, hook, spikes) * Dagger (1 red) * Fine clothing * Healer's kit (bandages, liquor, salves) * Jewellery * Lockpicks * Money * Oilskins (1 blue) * Provisions ==Maps== * Architectural drawings * Childhood memory * Familiar spirit * First-hand account * Legends, lore, and myth * Magnetic compass * Spellbook * Star charts * Trade and travel map * Treasure map ==Spells== * A guiding shadow * A warning vision * Arrows of flame * Bath of healing light * Lightnings at command * Luck for warriors * Passage unnoticed * Sensitivity to magic * Shelter of vines * Strength of ten ==People== * Healer * Priestess * Thug * Guardian * Elder * Boatmaker * Weaponsmith * Trader * Crafter * Imperial visitor * Refugee ==Default== Otherwise, everyone gets common clothing, including hat and clothes (gear), lunch and trail snacks (gear), flint, steel, and tinder, torches if called for (gear), and local lore (map). [[Category:RPGs]] [[Category:Siyavile]] 8ecf202bcd0bc1c504fcaf8ba24ef57ecbec9f48 2247 2244 2010-08-02T14:31:26Z Neil 2 /* Default */ wikitext text/x-wiki This is a description of the town of Siyavile, the base of our [http://www.lumpley.com/storming/ Storming the Wizard's Tower] game. =The town= [[Image:Siyavile-large-area.png|right|thumb|200px|The area around Siyavile]] This is a description of the town of Siyavile, the base of our Storming the Wizard's Tower game. Siyavile is a town on the end of a peninsula on an island. The area is tropical, with a short wet season (when it rains about 1m / yr) and a long dry season. The area has lots of rivers, which dry up at the end of the dry season. The island to the north of Siyavile is home of the large Epitira Empire. They've invaded and overthrown the kingdom that used to rule the eastern half of our island. Small battles continue to be fought in the desert between imperial soldiers and the small kingdoms on the western half of the island. There is an uneasy truce between the Empire and Siyavile, with some trade happening. But everyone knows that the Empire will try to take over the rest of the island, sooner or later. [[Image:Siyavile-local.png|right|thumb|200px|The town of Siyavile]] Siyavile is a town based on fishing and farming. The town has a moat and stone walls to defend itself. We catch the tastiest fish in the world. The people are really good sailors. There is a settlement outside the town, filled with refugees from the eastern side of the island. They have to stay outside because there's no room inside the walls for everyone. When the refugees arrived, there was a bit of a fight as they wanted to live in the town, but eventually they were forced to stay outside. Around the town is mostly savannah and farmland, with patches of jungle. There are lots of rivers crossing the plains near us. Nearby is a town of traders. We sell them our fish and we use the money to buy better ships from the Empire. People from the Empire are staying in the town to learn how we fish. Siyavile is ruled by King Tahir, a hereditary king. He appoints a council of about six people to advise him. Every month there is a big gathering where everyone can say what should happen next. The king lives in a three-floor mansion in the centre of town. The temple has lots of sections, one for each religion. There is lots of religious dancing, which is calm, not energetic. There are only a few dedicated soldiers in the town, who are more like police. When there is a war, everyone in the town will be expected to fight. The refugees are poor. Most people in the town are about as rich as each other, but there are a few rich ones and a few poor ones. =Names= {|- !Womens names !! Men's names |- | Adara<br> Ademia<br> Beryl<br> Charis<br> Corina<br> Delta<br> Eniyo<br> Filia<br> Ivanna<br> Kia<br> Nerine<br> Obella<br> Vanessa<br> | Abderus<br> Bedro<br> Garabed<br> Calchas<br> Capenus<br> Demophon<br> Hyponos<br> Idas<br> Irus<br> Jeno<br> Krikor<br> Mette<br> Yrjo<br> Zenon |} =Character types= ==Guardians== One of the soldiers/police from Siyavile. Don't choose this if you have low strength. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Hammerhands ==Trader (and explorer)== A traveller who sells fish and buys other goods, often in far-off places of the world. Don't choose this if you have low command or low perception. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Social training (or Danger Sense) ==Priest(ess)== You are a scholar of magic and wisdom. Don't choose this if you have both a low command and a low perception. * Choose 3 maps, 3 spells, 1 arms or gear, 1 from any list, and 1 person. * Ability: Initiation ==Imperial soldier== You have been trained by the Empire in how to fight. Don't choose this if you have low strength or low skill. * Choose 3 arms, 3 gear, 1 map or spell, 1 from any list, and 1 person. * Ability: Parrying (sword skill) ==Refugee== You have fled the Empire and now seek refuge in a strange land. Don't choose this if you have low Endurance. * Choose 3 arms, 3 maps, 1 gear or spells, 1 from any list, and 1 person. * Ability: Tireless ==Prince(ess)== Your father is the king. Don't choose this if you have both low command and low strength. * Choose 3 gear, 3 maps, 1 arms or spell, 1 from any list, and 1 person. * Ability: Tactician. =Lists= ==Arms== * Breastplate and helmet (2 blue) * Broadsword (2 red) * Chainmail (2 blue) * Handaxe (2 red) * Longbow (3 red, 2-handed, bulky) * Net (1 red and 1 blue) * Shield (2 blue) * Shortbow (2 red) * Shortsword (1 red) * Sling (1 red, 2-handed) * Spear (2 red) * Staff (1 red and 1 blue) * Throwing knives (1 red) * Trident (2 red) ==Gear== * Boat * Climbing gear (rope, hook, spikes) * Dagger (1 red) * Fine clothing * Healer's kit (bandages, liquor, salves) * Jewellery * Lockpicks * Money * Oilskins (1 blue) * Provisions ==Maps== * Architectural drawings * Childhood memory * Familiar spirit * First-hand account * Legends, lore, and myth * Magnetic compass * Spellbook * Star charts * Trade and travel map * Treasure map ==Spells== * A guiding shadow * A warning vision * Arrows of flame * Bath of healing light * Lightnings at command * Luck for warriors * Passage unnoticed * Sensitivity to magic * Shelter of vines * Strength of ten ==People== * Healer * Priestess * Thug * Guardian * Elder * Boatmaker * Weaponsmith * Trader * Crafter * Imperial visitor * Refugee ==Default== Otherwise, everyone gets common clothing, including hat and clothes (gear), lunch and trail snacks (gear), flint, steel, and tinder, torches if called for (gear), and local lore (map). [[Category:Siyavile]] 4994850a17bb16c21965f023030906ea3b3472b7 14 1597 2237 2010-07-31T16:07:52Z Neil 2 New page: Pages here relate to our game of [http://www.lumpley.com/storming/ Storming the Wizard's Tower]. wikitext text/x-wiki Pages here relate to our game of [http://www.lumpley.com/storming/ Storming the Wizard's Tower]. 2d5d8634042342128b4dc24e27b530793515b5cd 2240 2237 2010-07-31T16:25:25Z Neil 2 wikitext text/x-wiki Pages here relate to our game of [http://www.lumpley.com/storming/ Storming the Wizard's Tower]. [[Category:RPGs]] 71e1281c9978ceb65a03451d71b64ea75c6ecd83 6 1598 2238 2010-07-31T16:22:43Z Neil 2 Large area map of the surroundings of Siyavile. [[Category:Siyavile]] wikitext text/x-wiki Large area map of the surroundings of Siyavile. [[Category:Siyavile]] 43a0245baf0dfbc6fb512856b5565ea7f738e044 6 1599 2239 2010-07-31T16:24:03Z Neil 2 Local area around Siyavile [[Category:Siyavile]] wikitext text/x-wiki Local area around Siyavile [[Category:Siyavile]] ecce1b63b5cd824dc4544fcf90ecf27e0e1e077c 0 1542 2243 1906 2010-08-02T11:57:10Z Neil 2 Added bugfix details, formatting tidy wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# /etc/init.d/cupsys restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is only supported in HPLIP 2.8.10 and higher, but Ubuntu 8.04 only has HPLIP 2.8.2. So I need to update it. NB: Update the software before attaching the printer. * Remove the existing HPLIP installation. root@desktop:~# apt-get purge hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions] That's it! == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] 3e8f518ee1a97fef54487eac73bd2a498ccec866 0 1600 2245 2010-08-02T14:26:49Z Neil 2 [[Siyavile]] moved to [[Siyavile the town]] wikitext text/x-wiki #REDIRECT [[Siyavile the town]] 1e0b4d04b4afac625e17b2a0da6a35907bba7725 2246 2245 2010-08-02T14:31:07Z Neil 2 wikitext text/x-wiki Here is the information related to our game of [http://www.lumpley.com/storming/ Storming the Wizard's Tower]. You can read about: * [[Siyavile the town]], including all the Lists * The characters ** [[Atalanta]] the Guard ** [[Dynnamite]] the Princess ** [[Teucer]] the Explorer * Our adventures ** Coming soon! [[Category:Siyavile]] [[Category:RPGs]] c50be5646e869d4c4075046be10139d94b44d164 0 1601 2248 2010-08-02T14:40:38Z Neil 2 New page: ==Role== * Guard ==Statistics== * Arcane 3 [4] * Command 2 [3] * Endurance 6 [6] * Perception 4 [5] * Skill 6 [6] * Strength 6 [6] ==Arms== * Sword (2 red) * Shield (1 blue) * Helmet and... wikitext text/x-wiki ==Role== * Guard ==Statistics== * Arcane 3 [4] * Command 2 [3] * Endurance 6 [6] * Perception 4 [5] * Skill 6 [6] * Strength 6 [6] ==Arms== * Sword (2 red) * Shield (1 blue) * Helmet and breastplate (2 blue) ==Gear== * Dagger (1 red) * First aid kit * Boat ==Maps== * Treasure map ==People== * Someone from the Empire ==Abilities== * Hammerhands [[Category:Siyavile]] 49e3299b5d0bf9c5b33f2487b2da8bafbe8e9465 0 1602 2249 2010-08-02T14:40:44Z Neil 2 New page: ==Role== * Princess ==Statistics== * Arcane 1 [3] * Command 3 [4] * Endurance 6 [6] * Perception 2 [4] * Skill 5 [5] * Strength 1 [3] ==Arms== * Bow and arrows (2 red) ==Gear== * Dagger ... wikitext text/x-wiki ==Role== * Princess ==Statistics== * Arcane 1 [3] * Command 3 [4] * Endurance 6 [6] * Perception 2 [4] * Skill 5 [5] * Strength 1 [3] ==Arms== * Bow and arrows (2 red) ==Gear== * Dagger (1 red) * Money * Nice clothes ==Maps== * Star charts * Childhood memories * Legends, lore and myths ==People== * Ademia, a friendly fishergirl ==Abilities== * Tactician [[Category:Siyavile]] c7b54d0ac3dad9e471124a4a1728f0a9cfc8e77e 0 1603 2250 2010-08-02T14:47:45Z Neil 2 New page: ==Role== * Explorer ==Statistics== * Arcane 4 [5] * Command 1 [3] * Endurance 2 [4] * Perception 4 [6] * Skill 2 [4] * Strength 4 [5] ==Gear== * Fast boat * Provisions * Foreign spices =... wikitext text/x-wiki ==Role== * Explorer ==Statistics== * Arcane 4 [5] * Command 1 [3] * Endurance 2 [4] * Perception 4 [6] * Skill 2 [4] * Strength 4 [5] ==Gear== * Fast boat * Provisions * Foreign spices ==Maps== * Trade and travel maps * First-hand account * Familiar spirit ==People== * Kosmy, a priest ==Spells== * Passage unnoticed ==Abilities== * Danger Sense [[Category:Siyavile]] 8634e74d23717ed8d3da3321f162fb573a5d9072 0 1531 2251 2226 2010-08-03T09:54:37Z Neil 2 Added opening port 587 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki.dovecot.org Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki.dovecot.org/LDA Deliver] local delivery agent, to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are here. Dovecot, Deliver, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Basic Postfix with virtual users == Here we get Postfix running, by sending mail to other sites, and receiving mail and putting it in virtual users' mailboxes. I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /home/vmail/domain1.com/user1/ /user2/ /domain2.com/user1/ /user2/ /domain3.com/user1/ ...and so on. The <tt>vmail</tt> user will be the one used for email transport into and out of these virtual mailboxes. * Install postfix root@server:~# apt-get install postfix mailx * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update <tt>/etc/postfix/main.cf</tt> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <tt>/etc/postfix/vhosts</tt> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <tt>/etc/postfix/vmaps</tt> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <tt>/etc/postfix/vmaps</tt> is changed) * Create aliases for the users in <tt>/etc/postfix/valiases</tt> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <tt>/etc/postfix/valiases</tt> is changed) * Create <tt>/etc/aliases</tt> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <tt>telnet</tt> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <tt>mail</tt> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <tt>/etc/mailname</tt> mydomain.com * Install the packages: root@server:~# apt-get install amavis-new spamassassin clamav === ClamAV === * Add the ClamAV user to the Amavis group root@server:~# adduser clamav amavis The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <tt>/etc/default/spamassassin</tt> to activate the Spamassassin daemon. Change <tt>ENABLED=0</tt> to: ENABLED=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <tt>/etc/amavis/conf.d/15-content_filter_mode</tt> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <tt>/etc/amavis/conf.d/20-debian-defaults</tt> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <tt>/etc/postfix/main.cf</tt> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <tt>/etc/postfix/master.cf</tt> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <tt>recipient_delimiter</tt> option on Postfix is rather neat. Given a <tt>recipient_delimiter</tt> of '<tt>.</tt>', messages to <tt>user.ext1@domain1.com</tt> are handled in an interesting way. If <tt>user.ext1@domain1.com</tt> is a valid user, the message is put in their mailbox. If <tt>user.ext1@domain1.com</tt> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <tt>user@domain1.com</tt>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <tt>/etc/postfix/main.cf</tt> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <tt>/etc/postfix/header_checks</tt> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <tt>/etc/postfix/main.cf</tt> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 350bd6a5d660072acb28930e4fef3cb319924091 2267 2251 2011-01-21T10:17:59Z Neil 2 /* Amavis + ClamAV + SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki.dovecot.org Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki.dovecot.org/LDA Deliver] local delivery agent, to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are here. Dovecot, Deliver, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Basic Postfix with virtual users == Here we get Postfix running, by sending mail to other sites, and receiving mail and putting it in virtual users' mailboxes. I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /home/vmail/domain1.com/user1/ /user2/ /domain2.com/user1/ /user2/ /domain3.com/user1/ ...and so on. The <tt>vmail</tt> user will be the one used for email transport into and out of these virtual mailboxes. * Install postfix root@server:~# apt-get install postfix mailx * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update <tt>/etc/postfix/main.cf</tt> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <tt>/etc/postfix/vhosts</tt> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <tt>/etc/postfix/vmaps</tt> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <tt>/etc/postfix/vmaps</tt> is changed) * Create aliases for the users in <tt>/etc/postfix/valiases</tt> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <tt>/etc/postfix/valiases</tt> is changed) * Create <tt>/etc/aliases</tt> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <tt>telnet</tt> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <tt>mail</tt> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# apt-get install amavis-new spamassassin clamav-daemon root@server:~# apt-get install libnet-dns-perl libmail-spf-query-perl pyzor razor root@server:~# apt-get install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <tt>/etc/amavis/conf.d/20-debian-defaults</tt> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <tt>/etc/postfix/main.cf</tt> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <tt>/etc/postfix/master.cf</tt> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <tt>recipient_delimiter</tt> option on Postfix is rather neat. Given a <tt>recipient_delimiter</tt> of '<tt>.</tt>', messages to <tt>user.ext1@domain1.com</tt> are handled in an interesting way. If <tt>user.ext1@domain1.com</tt> is a valid user, the message is put in their mailbox. If <tt>user.ext1@domain1.com</tt> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <tt>user@domain1.com</tt>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <tt>/etc/postfix/main.cf</tt> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <tt>/etc/postfix/header_checks</tt> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <tt>/etc/postfix/main.cf</tt> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 37dd18c397a853cecab1fd0ecce3b09edb3b006e 0 1526 2252 2230 2010-08-04T03:30:08Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Mode switch key, and the 'right-click menu' key as the Compose key. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. For instance AltGr+5 will generate ½. <code>~/.Xmodmap</code> has the details. The Compose key allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 904f1bd6db86eca1f9dc8665c6a0a9854a212b44 2253 2252 2010-09-01T16:27:56Z Neil 2 /* Additional repositories */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Mode switch key, and the 'right-click menu' key as the Compose key. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. For instance AltGr+5 will generate ½. <code>~/.Xmodmap</code> has the details. The Compose key allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 159b18b3187d8329755daea83468e6eb29add516 2254 2253 2010-09-02T16:42:07Z Neil 2 /* Install and use Sun Java */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Mode switch key, and the 'right-click menu' key as the Compose key. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. For instance AltGr+5 will generate ½. <code>~/.Xmodmap</code> has the details. The Compose key allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 4eb91727d58ee33e8f489a705b0f3403acdd4b7e 2264 2254 2011-01-18T12:28:08Z Neil 2 Added noisy boot section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Mode switch key, and the 'right-click menu' key as the Compose key. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. For instance AltGr+5 will generate ½. <code>~/.Xmodmap</code> has the details. The Compose key allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] f536593dc6c9a504baed8edef61ea87be1049011 2265 2264 2011-01-21T08:16:24Z Neil 2 /* Noisy boot */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>apt-get autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/apt-get-autoclean</code>: #! /bin/sh # apt-get-autoclean Cron script to clear apt cache of unneeded files. # apt-get autoclean == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Mode switch key, and the 'right-click menu' key as the Compose key. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. For instance AltGr+5 will generate ½. <code>~/.Xmodmap</code> has the details. The Compose key allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 91f7e851a81cb4cbeac862b1163e34b86b038763 0 1577 2255 2225 2010-11-15T20:36:14Z Neil 2 /* Installation on Ubuntu Linux */ Updated with instructions for Ruby 1.9.2 on Ubuntu wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient-dev libruby-extras \ libyaml-ruby libzlib-ruby libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@desktop:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient-dev \ libyaml-ruby libzlib-ruby libruby-extras libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * (Optional) Installing Ruby 1.9.2 on Ubuntu, making a system-wide change with the <code>alternatives</code> system. ** Install Ruby 1.9.2 (still in the 1.9.1 package) root@desktop:~# apt-get install ruby1.9.1 ruby1.9.1-dev rubygems1.9.1 irb1.9.1 ri1.9.1 rdoc1.9.1 g++ ** Bring Ruby into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.8 500 \ --slave /usr/bin/ri ri /usr/bin/ri1.8 \ --slave /usr/bin/irb irb /usr/bin/irb1.8 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.8 ** Bring Ruby 1.9 into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 400 \ --slave /usr/bin/ri ri /usr/bin/ri1.9.1 \ --slave /usr/bin/irb irb /usr/bin/irb1.9.1 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.9.1 ** Choose which Ruby you want, system-wide root@desktop:~# update-alternatives --config ruby :and then choose Ruby 1.9.1 from the menu ** Check it works with root@desktop:~# ruby --version :(Instructions from [http://michalf.me/blog:make-ruby-1-9-default-on-ubuntu-9-10-karmic-koala Michal Frackowiak's blog]) :Or, if you want to install different versions on a per-user basis, or switch quickly between them, use [http://rvm.beginrescueend.com/ RVM]. * Build <code>gem</code> from [http://rubygems.org/pages/download source], to avoid it conflicting with <code>apt</code> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <code>sudo ruby rubygemsx.x.x/setup.rb</code>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :(The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both.) * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install sqlite3-ruby root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install sqlite3-ruby root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 635bb4a97f601bc8e413978df4cf3c82a616a0fa 0 1593 2256 2152 2011-01-05T16:48:08Z Neil 2 /* Set up gitweb */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 755044eeb7b32c14f6643557cc6d8f2b87d5f6cf 2257 2256 2011-01-18T10:01:53Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. this is a test edit. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 900f1322164e9636098dfe96694c58ef916564c8 2258 2257 2011-01-18T10:02:03Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:~# git clone git://eagain.net/gitosis.git root@server:~# cd gitosis root@server:~/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:~/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:~# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 755044eeb7b32c14f6643557cc6d8f2b87d5f6cf 2268 2258 2011-02-11T15:52:16Z Neil 2 /* Set up Gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 954d589206e014afe0f736132eea1ae609e190e4 2269 2268 2011-02-11T15:58:07Z Neil 2 /* Set up Gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# sudo -u git python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] c5b8b84c8f98452d71efce6bee45643901ae7c56 2270 2269 2011-02-11T16:00:08Z Neil 2 /* Set up Gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh test -f /usr/bin/git-daemon || exit 0 . /lib/lsb/init-functions GITDAEMON_OPTIONS="--reuseaddr --verbose --base-path=/home/git/repositories/ --detach" case "$1" in start) log_daemon_msg "Starting git-daemon" start-stop-daemon --start -c git:git --quiet --background \ --exec /usr/bin/git-daemon -- ${GITDAEMON_OPTIONS} log_end_msg $? ;; stop) log_daemon_msg "Stopping git-daemon" start-stop-daemon --stop --quiet --name git-daemon log_end_msg $? ;; restart) stop start ;; *) log_action_msg "Usage: /etc/init.d/git-daemon {start|stop|restart}" exit 2 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 63f04bac2e5b2a57874f5319703ea411a8cc5655 2271 2270 2011-02-11T16:56:15Z Neil 2 /* Set up git-daemon */ Revised git-daemon init script wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh NAME=git-daemon PIDFILE=/var/run/$NAME.pid DESC="the git daemon" DAEMON=/usr/lib/git-core/git-daemon DAEMON_OPTS="--base-path=/home/git/repositories --export-all --verbose --syslog --detach --pid-file=$PIDFILE --user=gitosis --group=nogroup" test -x $DAEMON || exit 0 [ -r /etc/default/git-daemon ] && . /etc/default/git-daemon . /lib/lsb/init-functions start_git() { start-stop-daemon --start --quiet --pidfile $PIDFILE \ --startas $DAEMON -- $DAEMON_OPTS } stop_git() { start-stop-daemon --stop --quiet --pidfile $PIDFILE rm -f $PIDFILE } status_git() { start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1 } case "$1" in start) log_begin_msg "Starting $DESC" start_git log_end_msg 0 ;; stop) log_begin_msg "Stopping $DESC" stop_git log_end_msg 0 ;; status) log_begin_msg "Testing $DESC: " if status_git then log_success_msg "Running" exit 0 else log_failure_msg "Not running" exit 1 fi ;; restart|force-reload) log_begin_msg "Restarting $DESC" stop_git sleep 1 start_git log_end_msg 0 ;; *) echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2 exit 1 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /var/www/git* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] b719fc664c36e9d2e2f79e99a40380b6e3d7cfba 2272 2271 2011-02-11T17:16:18Z Neil 2 /* Set up gitweb */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh NAME=git-daemon PIDFILE=/var/run/$NAME.pid DESC="the git daemon" DAEMON=/usr/lib/git-core/git-daemon DAEMON_OPTS="--base-path=/home/git/repositories --export-all --verbose --syslog --detach --pid-file=$PIDFILE --user=gitosis --group=nogroup" test -x $DAEMON || exit 0 [ -r /etc/default/git-daemon ] && . /etc/default/git-daemon . /lib/lsb/init-functions start_git() { start-stop-daemon --start --quiet --pidfile $PIDFILE \ --startas $DAEMON -- $DAEMON_OPTS } stop_git() { start-stop-daemon --stop --quiet --pidfile $PIDFILE rm -f $PIDFILE } status_git() { start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1 } case "$1" in start) log_begin_msg "Starting $DESC" start_git log_end_msg 0 ;; stop) log_begin_msg "Stopping $DESC" stop_git log_end_msg 0 ;; status) log_begin_msg "Testing $DESC: " if status_git then log_success_msg "Running" exit 0 else log_failure_msg "Not running" exit 1 fi ;; restart|force-reload) log_begin_msg "Restarting $DESC" stop_git sleep 1 start_git log_end_msg 0 ;; *) echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2 exit 1 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /usr/share/gitweb/* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] db00f903f89eca85806d01dede1f748e76dc4b2a 2273 2272 2011-02-11T19:26:02Z Neil 2 /* Creating new repositories in gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh NAME=git-daemon PIDFILE=/var/run/$NAME.pid DESC="the git daemon" DAEMON=/usr/lib/git-core/git-daemon DAEMON_OPTS="--base-path=/home/git/repositories --export-all --verbose --syslog --detach --pid-file=$PIDFILE --user=gitosis --group=nogroup" test -x $DAEMON || exit 0 [ -r /etc/default/git-daemon ] && . /etc/default/git-daemon . /lib/lsb/init-functions start_git() { start-stop-daemon --start --quiet --pidfile $PIDFILE \ --startas $DAEMON -- $DAEMON_OPTS } stop_git() { start-stop-daemon --stop --quiet --pidfile $PIDFILE rm -f $PIDFILE } status_git() { start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1 } case "$1" in start) log_begin_msg "Starting $DESC" start_git log_end_msg 0 ;; stop) log_begin_msg "Stopping $DESC" stop_git log_end_msg 0 ;; status) log_begin_msg "Testing $DESC: " if status_git then log_success_msg "Running" exit 0 else log_failure_msg "Not running" exit 1 fi ;; restart|force-reload) log_begin_msg "Restarting $DESC" stop_git sleep 1 start_git log_end_msg 0 ;; *) echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2 exit 1 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /usr/share/gitweb/* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == Perform these steps on the client machine. * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] f15de7334a32519da32d95d7ab5b86ba8bda3ea8 2274 2273 2011-02-11T19:26:32Z Neil 2 /* Creating new users in gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh NAME=git-daemon PIDFILE=/var/run/$NAME.pid DESC="the git daemon" DAEMON=/usr/lib/git-core/git-daemon DAEMON_OPTS="--base-path=/home/git/repositories --export-all --verbose --syslog --detach --pid-file=$PIDFILE --user=gitosis --group=nogroup" test -x $DAEMON || exit 0 [ -r /etc/default/git-daemon ] && . /etc/default/git-daemon . /lib/lsb/init-functions start_git() { start-stop-daemon --start --quiet --pidfile $PIDFILE \ --startas $DAEMON -- $DAEMON_OPTS } stop_git() { start-stop-daemon --stop --quiet --pidfile $PIDFILE rm -f $PIDFILE } status_git() { start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1 } case "$1" in start) log_begin_msg "Starting $DESC" start_git log_end_msg 0 ;; stop) log_begin_msg "Stopping $DESC" stop_git log_end_msg 0 ;; status) log_begin_msg "Testing $DESC: " if status_git then log_success_msg "Running" exit 0 else log_failure_msg "Not running" exit 1 fi ;; restart|force-reload) log_begin_msg "Restarting $DESC" stop_git sleep 1 start_git log_end_msg 0 ;; *) echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2 exit 1 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /usr/share/gitweb/* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == Perform these steps on the client machine. * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == Perform these steps on the client (desktop) machine. * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] cf97ff4ec9e8fdda92c8ccba1ef2a260a74a5fb1 2291 2274 2011-03-11T11:57:11Z Neil 2 /* Set up git-daemon */ Changed user in git-daemon startup script wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh NAME=git-daemon PIDFILE=/var/run/$NAME.pid DESC="the git daemon" DAEMON=/usr/lib/git-core/git-daemon DAEMON_OPTS="--base-path=/home/git/repositories --export-all --verbose --syslog --detach --pid-file=$PIDFILE --user=git --group=nogroup" test -x $DAEMON || exit 0 [ -r /etc/default/git-daemon ] && . /etc/default/git-daemon . /lib/lsb/init-functions start_git() { start-stop-daemon --start --quiet --pidfile $PIDFILE \ --startas $DAEMON -- $DAEMON_OPTS } stop_git() { start-stop-daemon --stop --quiet --pidfile $PIDFILE rm -f $PIDFILE } status_git() { start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1 } case "$1" in start) log_begin_msg "Starting $DESC" start_git log_end_msg 0 ;; stop) log_begin_msg "Stopping $DESC" stop_git log_end_msg 0 ;; status) log_begin_msg "Testing $DESC: " if status_git then log_success_msg "Running" exit 0 else log_failure_msg "Not running" exit 1 fi ;; restart|force-reload) log_begin_msg "Restarting $DESC" stop_git sleep 1 start_git log_end_msg 0 ;; *) echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2 exit 1 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /usr/share/gitweb/* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == Perform these steps on the client machine. * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == Perform these steps on the client (desktop) machine. * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] a27eec8338a9084ea84fe27f932dc7b9f192cc67 2292 2291 2011-03-11T13:57:42Z Neil 2 /* Creating new repositories in gitosis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh NAME=git-daemon PIDFILE=/var/run/$NAME.pid DESC="the git daemon" DAEMON=/usr/lib/git-core/git-daemon DAEMON_OPTS="--base-path=/home/git/repositories --export-all --verbose --syslog --detach --pid-file=$PIDFILE --user=git --group=nogroup" test -x $DAEMON || exit 0 [ -r /etc/default/git-daemon ] && . /etc/default/git-daemon . /lib/lsb/init-functions start_git() { start-stop-daemon --start --quiet --pidfile $PIDFILE \ --startas $DAEMON -- $DAEMON_OPTS } stop_git() { start-stop-daemon --stop --quiet --pidfile $PIDFILE rm -f $PIDFILE } status_git() { start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1 } case "$1" in start) log_begin_msg "Starting $DESC" start_git log_end_msg 0 ;; stop) log_begin_msg "Stopping $DESC" stop_git log_end_msg 0 ;; status) log_begin_msg "Testing $DESC: " if status_git then log_success_msg "Running" exit 0 else log_failure_msg "Not running" exit 1 fi ;; restart|force-reload) log_begin_msg "Restarting $DESC" stop_git sleep 1 start_git log_end_msg 0 ;; *) echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2 exit 1 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /usr/share/gitweb/* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == Perform these steps on the client machine. * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push :You will get an error remote: WARNING:gitosis.gitweb.set_descriptions:Cannot find 'repo_new' in '/home/git/repositories' remote: WARNING:gitosis.gitweb.generate_projects_list:Cannot find 'repo_new' in '/home/git/repositories' :This is normal. * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == Perform these steps on the client (desktop) machine. * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] a29359e77f3e8bc67f9c9ecb08202480b5502630 2293 2292 2011-03-11T14:29:49Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. == Set up a basic Git repository home == This is the place where your externally-accessbile Git repositories will reside. * Install the Git packages: root@server:~# apt-get install git-core gitweb git-svn fakeroot python-setuptools * Create the git user: root@server:~# adduser --system --shell /bin/sh --gecos 'git version control' --group --disabled-password --home /home/git git * Create somewhere to keep the repositories: root@server:~# sudo -u git mkdir /home/git/repositories * Allow SSH logins from trusted users. For each user who wants to log in, add their SSH key to the git user's <code>authorized_keys</code> file. (See [[Miscellaneous setup#Password-less_SSH_logins|Password-less SSH logins]] for details) * Allow SSH sessions to the git user. Edit <code>/etc/ssh/sshd_config</code> and add <code>git</code> to the <code>AllowUsers</code> line: AllowUsers user1 user2 git * Check that you can SSH into this user user@desktop:~$ ssh git@server :(you may need to restart the SSH daemon for it to pick up the new user) == Set up Gitosis == * Install the python setup tools needed for Gitosis root@server:~# apt-get install python-setuptools * Download and install Gitosis on the server: root@server:/home/git# sudo -u git git clone git://eagain.net/gitosis.git root@server:/home/git# cd gitosis root@server:/home/git/gitosis# python setup.py install * Copy the desktop user's public SSH key to the server and tell Gitosis about it: user@desktop:~$ scp ~/.ssh/id_rsa.pub user@server:/tmp/id_rsa.pub root@server:/home/git/gitosis# sudo -H -u git gitosis-init < /tmp/id_rsa.pub * Remove the original key from <code>/home/git/.ssh/authorized_keys</code> (the one without the 'autogenerated by gitosis' line) * Ensure that the gitosis hooks are executable: root@server:/home/git# sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update * On the desktop machine, checkout the Gitosis repository user@desktop:~$ git clone git@server:gitosis-admin.git user@desktop:~$ cd gitosis-admin * Update the <code>gitosis.conf</code> file as you wish (see below) then push the changes back to Gitosis user@desktop:~/gitosis-admin$ git commit -a -m "Some commit message" user@desktop:~/gitosis-admin$ git push == Set up git-daemon == Git-daemon is a simple tool that allows people to retrieve git repositories using the <code>git://</code> protocol. Unfortunately, the standard Ubuntu init script seems to have a bug, but it's easy enough to roll your own. * Create the file <code>/etc/init.d/git-daemon</code>: #!/bin/sh NAME=git-daemon PIDFILE=/var/run/$NAME.pid DESC="the git daemon" DAEMON=/usr/lib/git-core/git-daemon DAEMON_OPTS="--base-path=/home/git/repositories --export-all --verbose --syslog --detach --pid-file=$PIDFILE --user=git --group=nogroup" test -x $DAEMON || exit 0 [ -r /etc/default/git-daemon ] && . /etc/default/git-daemon . /lib/lsb/init-functions start_git() { start-stop-daemon --start --quiet --pidfile $PIDFILE \ --startas $DAEMON -- $DAEMON_OPTS } stop_git() { start-stop-daemon --stop --quiet --pidfile $PIDFILE rm -f $PIDFILE } status_git() { start-stop-daemon --stop --test --quiet --pidfile $PIDFILE >/dev/null 2>&1 } case "$1" in start) log_begin_msg "Starting $DESC" start_git log_end_msg 0 ;; stop) log_begin_msg "Stopping $DESC" stop_git log_end_msg 0 ;; status) log_begin_msg "Testing $DESC: " if status_git then log_success_msg "Running" exit 0 else log_failure_msg "Not running" exit 1 fi ;; restart|force-reload) log_begin_msg "Restarting $DESC" stop_git sleep 1 start_git log_end_msg 0 ;; *) echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2 exit 1 ;; esac exit 0 * Make it executable root@server:~# chomd a+x /etc/init.d/git-daemon * Make it a service and start it: root@server:~# update-rc.d git-daemon defaults root@server:~# /etc/init.d/git-daemon start * Don't forget to allow the git port (9418) through the firewall == Set up gitweb == We'll keep all the Gitweb files under <code>/home/git/gitweb</code>. The <code>gitweb.cgi</code> script will reside in <code>/var/www/cgi.site.domain.tld/</code>, to keep all the web files under <code>/var/www</code>. * Copy <code>gitweb.cgi</code> to the right place: root@server:~# cp /usr/lib/cgi-bin/gitweb.cgi /var/www/cgi-bin.site.domain.tld/ * Edit <code>/etc/gitweb.conf</code>: $my_uri = "<nowiki>http://site.domain.tld/gitweb</nowiki>"; $site_name = "site.domain.tld/gitweb"; # path to git projects (<project>.git) $projectroot = "/home/git/repositories"; # directory to use for temp files $git_temp = "/tmp"; # target of the home link on top of all pages $home_link = $my_uri; # html text to include at home page $home_text = "indextext.html"; # file with project list; by default, simply scan the projectroot dir. $projects_list = $projectroot; # stylesheet to use $stylesheet = "/gitweb/gitweb.css"; # logo to use $logo = "/gitweb/git-logo.png"; # the 'favicon' $favicon = "/gitweb/git-favicon.png"; * Edit <code>/etc/apache2/sites-available/site.domain.tld</code> to include this section as part of the <code>VirtualHost</code> section: RewriteEngine on RewriteRule ^/gitweb/([a-zA-Z0-9_\-]+\.git)/?(\?.*)?$ /cgi-bin/gitweb.cgi/$1 [L,PT] Alias /gitweb /home/git/gitweb <Directory /home/git/gitweb> Options Indexes FollowSymlinks ExecCGI DirectoryIndex /cgi-bin/gitweb.cgi AllowOverride None </Directory> <Directory /home/git> Order allow,deny Allow from all AllowOverride None </Directory> * Copy the .css and other files to <code>/home/git/gitweb</code> and fix permissions: root@server:~# cp /usr/share/gitweb/* /home/git/gitweb/ root@server:~# chown -R git:git /home/git/gitweb * Ensure existing repositories are accessbible to the www-data user root@server:~# chmod a+rx /home/git/repositories/repo1.git * Reload the Apache configuration root@server:~# /etc/init.d/apache2 reload You should now be able to browse to <code><nowiki>http://site.domain.tld/gitweb</nowiki></code> and see a list of publically-available repositories. == Creating new repositories in gitosis == Perform these steps on the client machine. * If needed, create a group of users by adding a <code>[group]</code> section to <code>gitosis.conf</code> * Add the repository name to the <code>writable</code> line in the group. * Push the updated <code>gitosis.conf</code> back to gitosis: user@desktop:~/gitosis-admin$ git commit -a -m "Added new repository repo1" user@desktop:~/gitosis-admin$ git push :You will get an error remote: WARNING:gitosis.gitweb.set_descriptions:Cannot find 'repo_new' in '/home/git/repositories' remote: WARNING:gitosis.gitweb.generate_projects_list:Cannot find 'repo_new' in '/home/git/repositories' :This is normal. * Create the new repository: user@desktop:~$ cd repo1 user@desktop:~/repo1$ git init user@desktop:~/repo1$ git add . user@desktop:~/repo1$ git remote add origin git@server:repo1.git :<add some files> user@desktop:~/repo1$ git push origin master:refs/heads/master * You will now be able to add a <code>[repo]</code> section to <code>gitosis.conf</code> and push it. Once you've put the repository on the server, you'll probably need to make it world-readable so that gitweb can pick it up root@server:~# chmod a+x /home/git/repositories/repo1.git == Creating new users in gitosis == Perform these steps on the client (desktop) machine. * Get the public SSH keys from the users you want to add (e.g. <code>alice.pub</code> and <code>bob.pub</code>). Note that the key filenames must have the .pub extension * Copy the files into the <code>keydir</code> directory of your local copy of the <code>gitosis</code> repository user@desktop:~$ cd /path/to/gitosis-admin user@desktop:~/gitosis-admin$ cp ~/alice.pub keydir/ user@desktop:~/gitosis-admin$ cp ~/bob.pub keydir/ user@desktop:~/gitosis-admin$ git add keydir/alice.pub keydir/bob.pub * Update the relevant <code>members</code> line in the <code>gitosis.conf</code> file: [group myteam] members = jdoe alice bob writable = repo1 * Commit and push the changes: user@desktop:~/gitosis-admin$ git commit -a -m "Granted Alice and Bob commit rights to Repo1" user@desktop:~/gitosis-admin$ git push * Alice and Bob can now clone the repo1 repository like so: git clone git@YOUR_SERVER_HOSTNAME:free_monkey.git :They also have commit rights. == Converting a Subversion repository to Git == The main problem is converting the Subversion branches and tags into Git branches. Create a clone of the SVN repository: user@desktop:~$ git svn clone -A authors.txt --trunk=path/to/trunk --branches=path/to/branches --tags=path/to/tags --no-metadata --username=user1 <nowiki>http://site.domain.tld/svn/svn_repo</nowiki> user@desktop:~$ cd svn_repo * Convert all the remote branches to local ones user@desktop:~/svn_repo$ git fetch . refs/remotes/*:refs/heads/* * Remove all the remote branches user@desktop:~/svn_repo$ for branch in `git branch -r`; do git branch -rd $branch; done user@desktop:~/svn_repo$ git branch -d trunk * Remove the SVN records user@desktop:~/svn_repo$ git config --remove-section svn-remote.svn user@desktop:~/svn_repo$ rm -rf .git/svn/ * Convert the 'tag' branches to proper tags user@desktop:~/svn_repo$ git-for-each-ref refs/heads/tags | cut -d / -f 4 | while read ref ; do git tag -a "$ref" -m "Tagging $tag" "refs/heads/tags/$ref" ; git branch -D "tags/$ref" ; done * Upload the repository to the git server: user@desktop:~/svn_repo$ git remote add origin git@git.domain.tld:repo.git user@desktop:~/svn_repo$ git push origin master == Sample <code>gitosis.conf</code> file == Here's a sample file [gitosis] # By default, gitweb and git-daemon won't process repositories gitweb = no daemon = no [group gitosis-admin] writable = gitosis-admin members = user@domain.tld [group groupone] members = user@domain.tld writable = repo1 repo2 [repo repo2] gitweb = yes daemon = yes description = A description of the repository == See also == * [http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way Hosting Git repositories, The Easy (and Secure) Way] * [http://batterypowered.wordpress.com/2008/07/04/deploying-a-git-repository-server-in-ubuntu/ Deploying a Git Repository Server in Ubuntu] * [http://blog.agdunn.net/?p=277 Installing Git and Gitosis on Ubuntu] * [http://www.howtoforge.com/setting-up-gitosis-on-ubuntu HowTo Forge: Setting Up Gitosis On Ubuntu] * [http://vafer.org/blog/20080115011320 From subversion to git (part1 – git)] * [http://vafer.org/blog/20080115011407 From subversion to git (part2 – gitweb)] * [http://vafer.org/blog/20080115011413 From subversion to git (part3 – gitosis)] * [http://maururu.net/2009/svn-to-git-take-2/ How to convert a SVN repository to Git] * [http://www.gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html Convert git-svn tag branches to real tags] [[Category:Server setup]] 23c052d23530a4ff7dc0b666801f486b5549c387 0 1534 2259 2192 2011-01-18T10:10:35Z Neil 2 /* Upgrading from previous versions */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/upload . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data upload root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# /etc/init.d/apache2 reload * Now got to <tt><nowiki>http://site.domain.tld/mediawiki/</nowiki></tt> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <tt>config/LocalSettings.php</tt> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <tt>$IP</tt> at the top of the file. * With the wiki set up, remove the <tt>config</tt> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <tt>LocalSettings.php</tt> file. The changes are marked below with <tt>###</tt> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <tt>/mediawiki/</tt> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <tt>/etc/apache2/sites-available/site.domain.tld</tt> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <tt><nowiki>http://site.domain.tld/Main_Page</nowiki></tt> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <tt>LocalSettings.php</tt>, modify the <tt>$wgEnableUploads</tt> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <tt>LocalSettings.php</tt> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <tt>LocalSettings.php</tt> file as shown. Add the lines just below the <tt>$wgEnableUploads</tt> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <tt>mediawiki-math</tt> pagkage is installed, it can be enabled with a simple edit of <tt>LocalSettings.php</tt>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <code>update.php</code> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <code>AdminSettings.php</code> file. Also, with a wiki farm, I'm not sure how to it will know which database to update. Instead, do the following: * Take a backup of the wiki database * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Create the <code>config</code> directory and copy the contents over from <code>/var/lib/mediawiki/config/</code> root@server:~# mkdir /var/www/site.domain.tld/mediawiki/config root@server:~# cp /var/lib/mediawiki/config/* /var/www/site.domain.tld/mediawiki/config/ root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/mediawiki/config * With a browser, visit <code><nowiki>http://site.domain.tld/mediawiki/index.php</nowiki></code> and follow the setup instructions * Remove any <code>LocalSettings.php</code> and the copy of the <code>config</code> directory * Rename <code>LocalSettings.php.old</code> to <code>LocalSettings.php</code> Old instructions: * Set up the new wiki, following the instructions as above, but without deleting the <tt>config</tt> directory * Overwrite the wiki database with the backed-up version * Remove the <tt>LocalSettings.php</tt> file * Go back to the wiki's config page. This will update the database schema * Once that's done, copy the new <tt>LocalSettings.php</tt> into the Mediawiki root and continue with the configuration. == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 54c256efe474a993a37e3b194dd0aaa49a68bccc 2262 2259 2011-01-18T11:29:50Z Neil 2 /* Upgrading from previous versions */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/upload . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data upload root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# /etc/init.d/apache2 reload * Now got to <tt><nowiki>http://site.domain.tld/mediawiki/</nowiki></tt> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <tt>config/LocalSettings.php</tt> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <tt>$IP</tt> at the top of the file. * With the wiki set up, remove the <tt>config</tt> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <tt>LocalSettings.php</tt> file. The changes are marked below with <tt>###</tt> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <tt>/mediawiki/</tt> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <tt>/etc/apache2/sites-available/site.domain.tld</tt> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <tt><nowiki>http://site.domain.tld/Main_Page</nowiki></tt> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <tt>LocalSettings.php</tt>, modify the <tt>$wgEnableUploads</tt> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <tt>LocalSettings.php</tt> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <tt>LocalSettings.php</tt> file as shown. Add the lines just below the <tt>$wgEnableUploads</tt> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <tt>mediawiki-math</tt> pagkage is installed, it can be enabled with a simple edit of <tt>LocalSettings.php</tt>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <code>update.php</code> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <code>AdminSettings.php</code> file. Also, with a wiki farm, I'm not sure how to it will know which database to update. Instead, do the following: * Take a backup of the wiki database * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Create the <code>config</code> directory and copy the contents over from <code>/var/lib/mediawiki/config/</code> root@server:~# mkdir /var/www/site.domain.tld/mediawiki/config root@server:~# cp /var/lib/mediawiki/config/* /var/www/site.domain.tld/mediawiki/config/ root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/mediawiki/config * With a browser, visit <code><nowiki>http://site.domain.tld/mediawiki/index.php</nowiki></code> and follow the setup instructions. Remember to include the password for the MySQL root user, so the update script can create or modify any tables it needs to. * Remove any <code>LocalSettings.php</code> and the copy of the <code>config</code> directory * Rename <code>LocalSettings.php.old</code> to <code>LocalSettings.php</code> == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] fec2e282febc116221d76eea530bee169f9e85b3 0 1532 2260 2228 2011-01-18T11:11:48Z Neil 2 /* Set up local delivery and sieve */ Dovecot now uses its own sieve plugin, not cmusieve wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This is all about the user-facing side of setting up a mail server. For information on setting up the server-facing side, see the [[Postfix server setup]] page. Also see the [[Webmail setup]] page. The basic source for setting up Postfix and Dovecot is the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois :(<tt>whois</tt> is needed in a moment for creating passwords) * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password :(If you want to change a user's password, delete their line from <code>/etc/dovecot/passwd</code> before running the <code>mkdovecotpasswd</code> script again.) * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <code>/etc/dovecot/dovecot.conf</code> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = sieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <code>/etc/postfix/main.cf</code> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <code>/etc/postfix/master.cf</code> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Create an an empty log file for <tt>deliver</tt> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 98c113da929ca134aeaf4695ddb59539be6c9da1 2263 2260 2011-01-18T11:54:42Z Neil 2 Added troubleshooting section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This is all about the user-facing side of setting up a mail server. For information on setting up the server-facing side, see the [[Postfix server setup]] page. Also see the [[Webmail setup]] page. The basic source for setting up Postfix and Dovecot is the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois :(<tt>whois</tt> is needed in a moment for creating passwords) * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password :(If you want to change a user's password, delete their line from <code>/etc/dovecot/passwd</code> before running the <code>mkdovecotpasswd</code> script again.) * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <code>/etc/dovecot/dovecot.conf</code> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = sieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <code>/etc/postfix/main.cf</code> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <code>/etc/postfix/master.cf</code> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Create an an empty log file for <tt>deliver</tt> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 432caacc141e5600343c79ab175630a81218f9b6 0 1515 2261 2170 2011-01-18T11:12:38Z Neil 2 Updated version number wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 10.04 LTS (adapted from 8.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 5f26417728f2e118f9110b9ea3b0d902576227c3 0 1533 2266 2204 2011-01-21T09:22:27Z Neil 2 /* Name-based virtual hosts */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === If you want one virtual host to be the default site served when nothing else matches, remove the <code>default</code> site from the list of available sites: root@server:~# a2dissite /etc/apache2/sites-available/default Then, for the site you want to be the default one, modify its configuration file to start with the line: <VirtualHost _default_:80> and remove the <code>ServerName</code> line. Then reload the configuration: root@server:~# /etc/init.d/apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] e8f81a5214b771a7c2ac2cd188fd53329d66b1cc 0 1 2275 2198 2011-03-06T21:36:17Z Neil 2 /* Neil */ Added sous vide wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. I also have a few [[downloadable programs]] (nothing exciting). I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 6931ab391c9123f8d85cefe4933c4cb2340dcc28 2297 2275 2011-03-11T20:02:10Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. I also have a few [[downloadable programs]] (nothing exciting). I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 60f483f386b08641dad2701d1ec7e6fdb9df58d1 0 1604 2276 2011-03-07T16:51:17Z Neil 2 Created page with 'I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous …' wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Equipment== To do sous vide cooking at home, you will need: * a slow cooker * a temperature controller * a thermocouple * an extension lead 0954d04d33867aaaa9fd8e345c42a50bec2697fe 2277 2276 2011-03-07T16:51:36Z Neil 2 wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Equipment== To do sous vide cooking at home, you will need: * a slow cooker * a temperature controller * a thermocouple * an extension lead [[Category:Sous Vide]] 36b8c848a0142f0c69018bb5d0c6d6ff506bc569 2278 2277 2011-03-07T16:52:01Z Neil 2 wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Equipment== To do sous vide cooking at home, you will need: * a slow cooker * a temperature controller * a thermocouple * an extension lead [[Category:Sous vide]] 276154cf66e67d276125644c8f12f368471d00bc 2279 2278 2011-03-07T20:24:40Z Neil 2 wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. ===Setup=== The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. ==See also== Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. [[Category:Sous vide]] 40eaf84a736336c2ed2df5e9a5e13bacdd4cac56 2280 2279 2011-03-07T20:25:29Z Neil 2 /* Equipment */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. ===Setup=== The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. ==See also== Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. [[Category:Sous vide]] 648e343f1da71f72035646ae8f05bd680f2467d0 2282 2280 2011-03-07T20:33:31Z Neil 2 /* Setup */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. ===Setup=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. ==See also== Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. [[Category:Sous vide]] 97b97a8c13f2eed5bc2688b4670d4760d3fe1dd3 2283 2282 2011-03-07T20:49:32Z Neil 2 wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. ===Setup=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 spec page of a TET 612 clone] (with a few more features than the TET 612) [[Category:Sous vide]] 3690d5cda523ff700619f187f76708da0f18d2ac 2284 2283 2011-03-07T21:24:40Z Neil 2 /* Setup */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. ===Setup=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 spec page of a TET 612 clone] (with a few more features than the TET 612) [[Category:Sous vide]] bf87c18e64abad23d9e4a71b140f140eedd30ba0 2285 2284 2011-03-07T21:26:08Z Neil 2 /* Setup */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. ===Setup=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 spec page of a TET 612 clone] (with a few more features than the TET 612) [[Category:Sous vide]] 2f7e7dea98004c50ce1acd3132f19e189d1d52e9 2286 2285 2011-03-07T21:38:27Z Neil 2 /* Setup */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. ===Setup=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Not used |- | CorF || 0 || Not used |} ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 spec page of a TET 612 clone] (with a few more features than the TET 612) [[Category:Sous vide]] 4b1c02968f167d21585a483df5b297be0d0ee6e0 2287 2286 2011-03-07T22:03:05Z Neil 2 /* Setup */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. ===Writing=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Not used |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the 'Set' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 spec page of a TET 612 clone] (with a few more features than the TET 612) [[Category:Sous vide]] 375b52ca9f64b5825111906a7c78cb4e4f6ab109 2288 2287 2011-03-07T22:06:24Z Neil 2 /* Writing */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Not used |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the 'Set' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 spec page of a TET 612 clone] (with a few more features than the TET 612) [[Category:Sous vide]] 50754e04cca02be541a46e1e83bbefa89c068785 2289 2288 2011-03-07T22:33:35Z Neil 2 wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==A more complex sous vide cooker== This is stage two, based on [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 the project from Make magazine]. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 58ebbb5ef5deb75cf52a7543ff753bd739c984e2 2290 2289 2011-03-07T22:46:05Z Neil 2 /* Wiring */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==A more complex sous vide cooker== This is stage two, based on [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 the project from Make magazine]. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] f5d66caf73ab268739f375dac2226a873fa9a071 2294 2290 2011-03-11T14:40:33Z Neil 2 /* See also */ wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==A more complex sous vide cooker== This is stage two, based on [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 the project from Make magazine]. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] f219d3fe04d4e1d44531e0849c9ede96731a5066 2295 2294 2011-03-11T20:00:44Z Neil 2 moved [[Sous vide setup]] to [[Simple sous vide setup]] wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. ==Very simple and cheap== This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==A more complex sous vide cooker== This is stage two, based on [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 the project from Make magazine]. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] f219d3fe04d4e1d44531e0849c9ede96731a5066 6 1605 2281 2011-03-07T20:30:13Z Neil 2 Wiring diagram for the simple sous vide controller wikitext text/x-wiki Wiring diagram for the simple sous vide controller 58d504da9c88b3df3ccf32642cea5c5e4c7ab5f9 14 1607 2298 2011-03-11T20:03:57Z Neil 2 Created page with 'Pages related to sous vide cooking' wikitext text/x-wiki Pages related to sous vide cooking 69e580810d663dd2c300802a715c8626cd8919ec 0 1608 2299 2011-03-11T20:04:15Z Neil 2 Created page with '==A more complex sous vide cooker== This is stage two, based on [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 the project from Make magazine]. [[Category:Sou…' wikitext text/x-wiki ==A more complex sous vide cooker== This is stage two, based on [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 the project from Make magazine]. [[Category:Sous vide]] 644bc443c390180c1a9ee29f963f9c0e0e708a49 0 1609 2300 2011-03-11T20:04:34Z Neil 2 Created page with 'Some recipies, and links to other recipies. [[Category:Sous vide]]' wikitext text/x-wiki Some recipies, and links to other recipies. [[Category:Sous vide]] 1cd6a117e718032851e289827ee7a4ab08dcaad2 2301 2300 2011-03-11T20:05:20Z Neil 2 wikitext text/x-wiki Some recipies, and links to other recipies. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] [[Category:Sous vide]] 0533eea7aeab1f7d51adb01140b3f7bea369080a 0 1604 2302 2295 2011-03-11T20:05:46Z Neil 2 wikitext text/x-wiki This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] b5856deea3cd73d8350172c74745bb215d6ced20 2304 2302 2011-03-11T20:06:42Z Neil 2 wikitext text/x-wiki Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket. Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's cheap and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay. * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket. * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] a5625781bd6ad9747cc819ac2cff7c20d3130ffe 2305 2304 2011-03-11T20:19:16Z Neil 2 /* Equipment */ wikitext text/x-wiki Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket (under £20.00). Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's not too expensive (£26.00) and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay (£5.00). * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket (about £3.00). * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 4cdc2e79f99a6b9581ddec66066517fc8e485f2c 2316 2305 2011-05-27T08:35:18Z Neil 2 /* Controller tuning */ wikitext text/x-wiki Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket (under £20.00). Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's not too expensive (£26.00) and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay (£5.00). * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket (about £3.00). * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 83407341e42b18158696d7e75c1e94eb82d66a3e 2317 2316 2011-05-27T08:35:53Z Neil 2 /* Controller tuning */ wikitext text/x-wiki Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket (under £20.00). Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's not too expensive (£26.00) and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay (£5.00). * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket (about £3.00). * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. It also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 6ca96bd3a694186b082f1ce0c0bb07465b6654cc 2330 2317 2011-05-27T14:10:36Z Neil 2 /* Equipment */ wikitext text/x-wiki Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ===Equipment=== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket (under £20.00). Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's not too expensive (£26.00) and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay (£5.00). * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket (about £3.00). * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. The controller also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ===Wiring=== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ===Controller parameters=== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] efaba405fc05555217f525b5a9c53e470158e744 2341 2330 2011-05-28T07:29:34Z Neil 2 wikitext text/x-wiki Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ==Equipment== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket (under £20.00). Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's not too expensive (£26.00) and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay (£5.00). * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket (about £3.00). * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. The controller also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ==Wiring== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ==Controller tuning== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 327161870ccdeea42df885342b2202f35e52bc3a 14 1607 2303 2298 2011-03-11T20:06:30Z Neil 2 wikitext text/x-wiki I was inspired to try [http://en.wikipedia.org/wiki/Sous-vide sous vide cooking] after receiving the book [http://www.cookingforgeeks.com/ Cooking for Geeks] for Christmas. Sous vide is cooking food in a temperature-controlled water bath, with the food in vacuum-sealed bags. Cooking times range from 30 minutes to 36 hours or more. c4a1bca752148224efda87e21efcc9521aea9d04 0 1609 2306 2301 2011-03-11T20:25:51Z Neil 2 wikitext text/x-wiki Some recipies, and links to other recipies. ===Chicken breasts=== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] [[Category:Sous vide]] cbcebb76185f361238bd4f0baed15e76bd5ed599 2308 2306 2011-04-11T08:32:15Z Neil 2 wikitext text/x-wiki Some recipies, and links to other recipies. ==Chicken breasts== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] [[Category:Sous vide]] e6f57d88b68738179d42eb31805f3239faae34a9 2309 2308 2011-04-13T08:23:22Z Neil 2 wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 58⁰C for 60 minutes. ==Chicken breasts== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] [[Category:Sous vide]] cc28162f0c56598c89a793f25630bf89c443761d 2310 2309 2011-04-23T14:12:59Z Neil 2 Added glazed carrots recipe wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 58⁰C for 60 minutes. ==Chicken breasts== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==Glazed carrots== * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] [[Category:Sous vide]] ce1f399262ccc96d51a18320e12f5a4a482674de 2313 2310 2011-05-25T12:59:12Z Neil 2 /* Glazed carrots */ Added warning wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 58⁰C for 60 minutes. ==Chicken breasts== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==Glazed carrots== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] [[Category:Sous vide]] cae0a4d4ad8d04015be12c285f02e6b8f1605aaf 2314 2313 2011-05-25T12:59:49Z Neil 2 /* See also */ wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 58⁰C for 60 minutes. ==Chicken breasts== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==Glazed carrots== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] [[Category:Sous vide]] 04f2cebaf4f7471f9350e538a6067169c35321aa 2319 2314 2011-05-27T09:25:33Z Neil 2 /* See also */ wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 58⁰C for 60 minutes. ==Chicken breasts== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==Glazed carrots== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] [[Category:Sous vide]] ccdfdff164fbf971e831ff60fd0c7dd7847b405f 2325 2319 2011-05-27T13:40:47Z Neil 2 wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 58⁰C for 60 minutes. ==Chicken breasts== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) ==Retrograde starch mashed potatoes (untried)== * Slice potatoes and cook at 70⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally * Mash, and it won't end up gloopy ==Aspargus (untried)== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] [[Category:Sous vide]] e3704a0541c071607bdc6e1296c1e9d2167f92cb 2326 2325 2011-05-27T13:47:32Z Neil 2 /* Retrograde starch mashed potatoes (untried) */ wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 58⁰C for 60 minutes. ==Chicken breasts== 63⁰C, 60 minutes. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse thoroughly under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) ==Retrograde starch mashed potatoes (untried)== * Slice potatoes and cook at 70⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy ==Aspargus (untried)== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] [[Category:Sous vide]] 786562fdd7b93277a82074ae45065d1bb8bfbf71 2345 2326 2011-05-31T22:42:56Z Neil 2 Added verdicts wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes (untried)== * Slice potatoes and cook at 70⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] [http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] [http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (US short ribs are UK thin ribs) [[Category:Sous vide]] 6ef32cc4520fca65d2d842575fe551653c39b207 2347 2345 2011-06-06T21:33:50Z Neil 2 Added mutton wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes (untried)== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==Mutton (untried)== No aromatics beyond a little salt and pepper. 70⁰C for 24 hours. From [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery], if you can get the link to work. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] [http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] [http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (US short ribs are UK thin ribs) [[Category:Sous vide]] 535f361d44558bf9c32e0a27fbdcde267faf11f0 2348 2347 2011-06-06T21:34:03Z Neil 2 /* Retrograde starch mashed potatoes (untried) */ wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==Mutton (untried)== No aromatics beyond a little salt and pepper. 70⁰C for 24 hours. From [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery], if you can get the link to work. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] [http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] [http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (US short ribs are UK thin ribs) [[Category:Sous vide]] 57342c406826584960f5ff989d744540cf6c427a 2349 2348 2011-06-06T21:41:23Z Neil 2 /* See also */ wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==Mutton (untried)== No aromatics beyond a little salt and pepper. 70⁰C for 24 hours. From [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery], if you can get the link to work. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] 013ffffc2b01ea76e8c228372e16c2972504f3ac 2350 2349 2011-06-06T21:42:12Z Neil 2 wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton (untried)== No aromatics beyond a little salt and pepper. 70⁰C for 24 hours. From [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery], if you can get the link to work. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] 3c3b2c65211bbdaa6ea7e98d0529281233740863 0 1 2307 2297 2011-03-12T16:56:00Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. faa5b8e47a6c65cc2345be2423a0c1e8483cedf1 0 1544 2311 2201 2011-05-02T09:35:33Z Neil 2 /* Chrooting the nameservers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld NS desktop.domain.tld ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/named root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run root@server:~# chown -R bind:bind /chroot/bind/var/cache * Modify the file <code>/etc/default/bind9</code> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * '''For Ubuntu 8.04:''' Modify <code>/etc/default/syslogd</code> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * '''For Ubuntu 10.04:''' Create <code>/etc/rsyslog.d/bind-chroot.conf</code> to log messages from the chrooted BIND: $AddUnixListenSocket /chroot/bind/dev/log * Make AppArmor allow BIND access to the chroot jail. Modify <code>/etc/apparmor.d/usr.sbin.named</code> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/lib/bind/ rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/cache/bind/ rw, /chroot/bind/var/run/named/named.pid w, /chroot/bind/var/run/named/session.key w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, system logger, and BIND: root@server:~# /etc/init.d/apparmor restart ** '''For Ubuntu 8.04:''' root@server:~# /etc/init.d/sysklogd restart ** '''For Ubuntu 10.04:''' root@server:~# service rsyslog restart * For both: root@server:~# /etc/init.d/bind9 restart * Check <code>/var/log/syslog</code> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 003de72466e5c07dd128b789386f930967ec3c4c 2312 2311 2011-05-02T09:39:28Z Neil 2 /* Getting the servers running */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/named root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run root@server:~# chown -R bind:bind /chroot/bind/var/cache * Modify the file <code>/etc/default/bind9</code> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * '''For Ubuntu 8.04:''' Modify <code>/etc/default/syslogd</code> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * '''For Ubuntu 10.04:''' Create <code>/etc/rsyslog.d/bind-chroot.conf</code> to log messages from the chrooted BIND: $AddUnixListenSocket /chroot/bind/dev/log * Make AppArmor allow BIND access to the chroot jail. Modify <code>/etc/apparmor.d/usr.sbin.named</code> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/lib/bind/ rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/cache/bind/ rw, /chroot/bind/var/run/named/named.pid w, /chroot/bind/var/run/named/session.key w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, system logger, and BIND: root@server:~# /etc/init.d/apparmor restart ** '''For Ubuntu 8.04:''' root@server:~# /etc/init.d/sysklogd restart ** '''For Ubuntu 10.04:''' root@server:~# service rsyslog restart * For both: root@server:~# /etc/init.d/bind9 restart * Check <code>/var/log/syslog</code> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 73506d49a81c9ceaba27bf67f34aff0dea512477 0 1608 2315 2299 2011-05-26T18:38:50Z Neil 2 /* A more complex sous vide cooker */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. ==Parts== * heaters * box * thermocouple * controller * relay ==Materials== * glue * perspex ==A more complex sous vide cooker== This is stage two, based on [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 the project from Make magazine]. [[Category:Sous vide]] 024b3e43b610204639c2c2596f0399c0f541afaa 2318 2315 2011-05-27T08:40:38Z Neil 2 wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole heater much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the frame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me very wary of using the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the lid and place the cooker over that. The lid supports the box and the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. There is no need to adjust the 'ot' parameter, as it's controlling an SSR. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 77cbd31c20ba2c064aec5643bfcdad6a84abf088 2320 2318 2011-05-27T12:08:48Z Neil 2 /* Controller tuning */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole heater much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the frame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me very wary of using the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the lid and place the cooker over that. The lid supports the box and the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. There is no need to adjust the 'ot' parameter, as it's controlling an SSR. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 43 |- | d || 10 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 661b174eb6aaec9ae75e578db51d673d0ea6103d 2321 2320 2011-05-27T12:09:02Z Neil 2 /* Controller tuning */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole heater much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the frame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me very wary of using the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the lid and place the cooker over that. The lid supports the box and the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. There is no need to adjust the 'ot' parameter, as it's controlling an SSR. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.04 |- | I || 43 |- | d || 10 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] e7469932ba81e4bcca768d0e375150e6b1d887b8 2322 2321 2011-05-27T13:21:43Z Neil 2 /* Controller tuning */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole heater much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the frame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me very wary of using the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the lid and place the cooker over that. The lid supports the box and the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. There is no need to adjust the 'ot' parameter, as it's controlling an SSR. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 51a48ea914005bc20f389bda4ad5346f9239a966 2323 2322 2011-05-27T13:27:38Z Neil 2 /* Controller tuning */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole heater much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the frame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me very wary of using the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the lid and place the cooker over that. The lid supports the box and the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 3912b38a13511edc20e62d4306db88bf2d49027a 2324 2323 2011-05-27T13:28:57Z Neil 2 /* See also */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole heater much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the frame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me very wary of using the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the lid and place the cooker over that. The lid supports the box and the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 6d3797b58d1382f0188f92dc9e19257d91aeda7d 2327 2324 2011-05-27T14:04:06Z Neil 2 /* Parts */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the frame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me very wary of using the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the lid and place the cooker over that. The lid supports the box and the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 156a972fe288fb8cd8b3bfc12f4d6e7c60beaa22 2328 2327 2011-05-27T14:05:57Z Neil 2 /* Construction tips */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me very wary of using the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the lid and place the cooker over that. The lid supports the box and the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 5197e5ad55b12ee9fdb733ed1f904f8da936b4a7 2329 2328 2011-05-27T14:07:19Z Neil 2 /* Construction tips */ wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 9e7574ecfa1ba7f9e680e675b4798c3b81a17ae6 2331 2329 2011-05-27T21:25:41Z Neil 2 moved [[Complex sous vide setup]] to [[Advanced sous vide setup]]:&#32;Better title wikitext text/x-wiki The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 9e7574ecfa1ba7f9e680e675b4798c3b81a17ae6 2337 2331 2011-05-27T21:32:53Z Neil 2 Added photos wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The advanced sous vide cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the advanced sous vide cooker, showing a rats' nest of wires]] [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the advanced sous vide cooker, showing the arrangement of the parts that go into the water]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 15e56cf58ea34d0bb86989658c0147d5be94903f 2339 2337 2011-05-28T07:21:34Z Neil 2 Added wiring diagram wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== To follow. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== * Calibration of the thermocouple The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 3872fd1d6ec8c89f71d4cb8d6968d579514c9c00 2340 2339 2011-05-28T07:28:04Z Neil 2 wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. (If you adjusted the 'ot' parameter after using the controller's built-in relay, you'll need to reset it to 2 before tuning.) The controller will move the water temperature above and below the target three times, so it could take a few hours to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 485 |- | d || 121 |- | Souf || 0.2 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 8dec33cdaf871fdcb513345ecdcc0e7503e8af09 2343 2340 2011-05-31T17:06:25Z Neil 2 /* Controller tuning */ wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] bfa91db5211a1b7613f42bc1c4d9d295bb80be32 2351 2343 2011-06-22T09:29:33Z Neil 2 /* See also */ wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] a7040083500144eacc04e34635eec5b6905888e7 2352 2351 2011-06-22T09:42:55Z Neil 2 /* Parts */ wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. * '''Circulation pump''': Most standard aquarium power heads are only rated up to 35⁰C and quickly fail at sous vide temperatures of 60⁰C and above. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 8265cf4c1896ff088b9a7586111b94c7b877eef2 6 1611 2333 2011-05-27T21:26:30Z Neil 2 The advanced sous vide cooker. wikitext text/x-wiki The advanced sous vide cooker. 81d84561ddf027fce6d4b570fa3cf01999413a0d 2334 2333 2011-05-27T21:27:03Z Neil 2 Added category link wikitext text/x-wiki The advanced sous vide cooker. [[Category:Sous vide]] fe4f462d6ea10edcbe92e1086d799fcdf74c0fef 6 1612 2335 2011-05-27T21:27:46Z Neil 2 Inside the sous vide cooker controls. [[Category:Sous vide]] wikitext text/x-wiki Inside the sous vide cooker controls. [[Category:Sous vide]] e65b2d1d5f19f5da02e913ebd5b0702c757315d6 6 1613 2336 2011-05-27T21:28:37Z Neil 2 The underside of the advanced sous vide cooker, showing the arrangement of all the parts that go into the water. [[Category:Sous vide]] wikitext text/x-wiki The underside of the advanced sous vide cooker, showing the arrangement of all the parts that go into the water. [[Category:Sous vide]] 07beb4af457ad57dd6fe2e93e22b27043e1de2b7 6 1614 2338 2011-05-28T07:17:42Z Neil 2 Wiring guide for the advanced sous vide cooker [[Category:Sous vide]] wikitext text/x-wiki Wiring guide for the advanced sous vide cooker [[Category:Sous vide]] c238553e9a47837379c4e57a6e203964dcee03fa 0 1588 2342 2129 2011-05-31T14:46:40Z Neil 2 /* Change the port sshd listens on */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. == Changes to config file == But before that, there are a couple of changes that need to be made to the SSH server's <code>/etc/ssh/sshd_config</code> file. Disable root logins by changing the line to this # Authentication: PermitRootLogin no to prevent attempts to log in as root. Add the line AllowUsers user1 user2 user3 to only allow SSH access to these users. Then restart the SSH server: root@server:~# /etc/init.d/ssh restart == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 (If you want <code>sshd</code> to listen on several ports, give several <code>Port</code> lines.) To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict connections to those machines using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [[Firewall setup]] page [[Category:Server setup]] 513fc2c495869aeeb23c7cc45d2c7b05ecaf5881 6 1605 2344 2281 2011-05-31T22:27:19Z Neil 2 Added category link wikitext text/x-wiki Wiring diagram for the simple sous vide controller. [[Category:Sous vide]] b29e1830822f84109fdf8bfbe205b56082a57851 0 1615 2346 2011-05-31T22:46:14Z Neil 2 Created page with '* Freeze liquid marinades before using them. Liquids will just get sucked into the clamp vacuum sealer and damage it. * Double seal all bags (i.e. after the first evacuate-and-se…' wikitext text/x-wiki * Freeze liquid marinades before using them. Liquids will just get sucked into the clamp vacuum sealer and damage it. * Double seal all bags (i.e. after the first evacuate-and-seal, create an additional seal at both ends of the bag, inside the original seals. * If you're cooking for more than a few hours, double-bag the food. * Put a handful of glass nuggets (of the type used in crafts and flower arranging) in the bottom of a bag before sealing it. That will stop the bags from floating. The buoyancy of the food will keep it floating vertically, allowing better water circulation around the food. * Before cooking, scald the sealed bags of food for 30 seconds in boiling water. That won't cook the food, but will destroy any bacteria on its surface. * Ensure that the food is no more than 10% of the water volume. [[Category:Sous vide]] 661c3119d1b38562470f9f0026ac2caa48501565 0 1608 2353 2352 2011-06-22T10:08:37Z Neil 2 /* Parts */ wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of had screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. * '''Circulation pump''': Most standard aquarium power heads are only rated up to 35⁰C and quickly fail at sous vide temperatures of 60⁰C and above. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** [http://cgi.ebay.com/DC-6V-Mini-Submersible-Water-Pump-35-GPH-P-32A-/260688654675?pt=LH_DefaultDomain_0&hash=item3cb2413153#ht_3239wt_1141 DC 6V Mini Submersible Water Pump 35 GPH (P-32A)], a submersible pump rated up to 80⁰C ** [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 DC 12V Electric Submersible Water Pump 95 GPH (P-38B)], a submersible pump rated up to 80⁰C ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 5dbc1850b41dbd33c821b25b500a0b04a43b352c 2355 2353 2011-06-22T15:23:39Z Neil 2 /* Parts */ wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': Most standard aquarium power heads are only rated up to 35⁰C and quickly fail at sous vide temperatures of 60⁰C and above. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** [http://cgi.ebay.com/DC-6V-Mini-Submersible-Water-Pump-35-GPH-P-32A-/260688654675?pt=LH_DefaultDomain_0&hash=item3cb2413153#ht_3239wt_1141 DC 6V Mini Submersible Water Pump 35 GPH (P-32A)], a submersible pump rated up to 80⁰C ** [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 DC 12V Electric Submersible Water Pump 95 GPH (P-38B)], a submersible pump rated up to 80⁰C ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 32f50d4214d7d86d88882a891f730fba389d71b4 2379 2355 2011-09-25T09:39:46Z Neil 2 moved [[Advanced sous vide setup]] to [[Sous vide setup mk 2]] wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[simple sous vide setup]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': Most standard aquarium power heads are only rated up to 35⁰C and quickly fail at sous vide temperatures of 60⁰C and above. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** [http://cgi.ebay.com/DC-6V-Mini-Submersible-Water-Pump-35-GPH-P-32A-/260688654675?pt=LH_DefaultDomain_0&hash=item3cb2413153#ht_3239wt_1141 DC 6V Mini Submersible Water Pump 35 GPH (P-32A)], a submersible pump rated up to 80⁰C ** [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 DC 12V Electric Submersible Water Pump 95 GPH (P-38B)], a submersible pump rated up to 80⁰C ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 32f50d4214d7d86d88882a891f730fba389d71b4 2381 2379 2011-09-25T09:40:15Z Neil 2 wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 1]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Simple sous vide setup]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Simple sous vide setup]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': Most standard aquarium power heads are only rated up to 35⁰C and quickly fail at sous vide temperatures of 60⁰C and above. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** [http://cgi.ebay.com/DC-6V-Mini-Submersible-Water-Pump-35-GPH-P-32A-/260688654675?pt=LH_DefaultDomain_0&hash=item3cb2413153#ht_3239wt_1141 DC 6V Mini Submersible Water Pump 35 GPH (P-32A)], a submersible pump rated up to 80⁰C ** [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 DC 12V Electric Submersible Water Pump 95 GPH (P-38B)], a submersible pump rated up to 80⁰C ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] f8777286ad525b54ed5cfb3c8412dbe9e3b779a3 2383 2381 2011-09-25T09:42:11Z Neil 2 /* Parts */ wikitext text/x-wiki [[File:Sous-vide-cooker.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 1]] is good for gettng started, but it's limited in the amount of food you can cook in it. A larger-volume sous vide cooker is easy to make but a bit more involved. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': Most standard aquarium power heads are only rated up to 35⁰C and quickly fail at sous vide temperatures of 60⁰C and above. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** [http://cgi.ebay.com/DC-6V-Mini-Submersible-Water-Pump-35-GPH-P-32A-/260688654675?pt=LH_DefaultDomain_0&hash=item3cb2413153#ht_3239wt_1141 DC 6V Mini Submersible Water Pump 35 GPH (P-32A)], a submersible pump rated up to 80⁰C ** [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 DC 12V Electric Submersible Water Pump 95 GPH (P-38B)], a submersible pump rated up to 80⁰C ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 2a378e61a5c8431fc1c01d7195655a643cd24f6c 6 1614 2354 2338 2011-06-22T10:43:10Z Neil 2 uploaded a new version of "[[File:Tet612-sous-vide-wiring-advanced.png]]":&#32;Corrected SSR terminal wiring wikitext text/x-wiki Wiring guide for the advanced sous vide cooker [[Category:Sous vide]] c238553e9a47837379c4e57a6e203964dcee03fa 0 1577 2356 2255 2011-06-29T16:10:55Z Neil 2 wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient-dev libruby-extras \ libyaml-ruby libzlib-ruby libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@desktop:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient-dev \ libyaml-ruby libzlib-ruby libruby-extras libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * (Optional) Installing Ruby 1.9.2 on Ubuntu, making a system-wide change with the <code>alternatives</code> system. ** Install Ruby 1.9.2 (still in the 1.9.1 package) root@desktop:~# apt-get install ruby1.9.1 ruby1.9.1-dev rubygems1.9.1 irb1.9.1 ri1.9.1 rdoc1.9.1 g++ ** Bring Ruby into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.8 500 \ --slave /usr/bin/ri ri /usr/bin/ri1.8 \ --slave /usr/bin/irb irb /usr/bin/irb1.8 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.8 ** Bring Ruby 1.9 into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 400 \ --slave /usr/bin/ri ri /usr/bin/ri1.9.1 \ --slave /usr/bin/irb irb /usr/bin/irb1.9.1 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.9.1 ** Choose which Ruby you want, system-wide root@desktop:~# update-alternatives --config ruby :and then choose Ruby 1.9.1 from the menu ** Check it works with root@desktop:~# ruby --version :(Instructions from [http://michalf.me/blog:make-ruby-1-9-default-on-ubuntu-9-10-karmic-koala Michal Frackowiak's blog]) :Or, if you want to install different versions on a per-user basis, or switch quickly between them, use [http://rvm.beginrescueend.com/ RVM]. * Build <code>gem</code> from [http://rubygems.org/pages/download source], to avoid it conflicting with <code>apt</code> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <code>sudo ruby rubygemsx.x.x/setup.rb</code>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :(The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both.) * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install sqlite3-ruby root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install sqlite3-ruby root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios user@desktop:~$ gem install rails rake rcov rspec gemcutter git jeweler capistrano mysql sqlite3 git shoulda :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] cc6093efa2c271fed030fbe34527a2ad2926fba0 2357 2356 2011-06-30T15:06:11Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient-dev libruby-extras \ libyaml-ruby libzlib-ruby libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@desktop:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient-dev \ libyaml-ruby libzlib-ruby libruby-extras libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * (Optional) Installing Ruby 1.9.2 on Ubuntu, making a system-wide change with the <code>alternatives</code> system. ** Install Ruby 1.9.2 (still in the 1.9.1 package) root@desktop:~# apt-get install ruby1.9.1 ruby1.9.1-dev rubygems1.9.1 irb1.9.1 ri1.9.1 rdoc1.9.1 g++ ** Bring Ruby into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.8 500 \ --slave /usr/bin/ri ri /usr/bin/ri1.8 \ --slave /usr/bin/irb irb /usr/bin/irb1.8 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.8 ** Bring Ruby 1.9 into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 400 \ --slave /usr/bin/ri ri /usr/bin/ri1.9.1 \ --slave /usr/bin/irb irb /usr/bin/irb1.9.1 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.9.1 ** Choose which Ruby you want, system-wide root@desktop:~# update-alternatives --config ruby :and then choose Ruby 1.9.1 from the menu ** Check it works with root@desktop:~# ruby --version :(Instructions from [http://michalf.me/blog:make-ruby-1-9-default-on-ubuntu-9-10-karmic-koala Michal Frackowiak's blog]) :Or, if you want to install different versions on a per-user basis, or switch quickly between them, use [http://rvm.beginrescueend.com/ RVM]. * Build <code>gem</code> from [http://rubygems.org/pages/download source], to avoid it conflicting with <code>apt</code> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <code>sudo ruby rubygemsx.x.x/setup.rb</code>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :(The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both.) * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install sqlite3-ruby root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install sqlite3-ruby root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios ===RVM== If using RVM to keep different Ruby versions around, you need to install the Readline development headers first: root@desktop:~# apt-get install libreadline-dev user@desktop:~$ gem install rails rake rcov rspec gemcutter git jeweler capistrano mysql sqlite3 git shoulda :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 901f51f41f89d110abaa968354d3c76765472dd1 2358 2357 2011-06-30T15:06:22Z Neil 2 /* =RVM */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient-dev libruby-extras \ libyaml-ruby libzlib-ruby libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@desktop:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient-dev \ libyaml-ruby libzlib-ruby libruby-extras libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * (Optional) Installing Ruby 1.9.2 on Ubuntu, making a system-wide change with the <code>alternatives</code> system. ** Install Ruby 1.9.2 (still in the 1.9.1 package) root@desktop:~# apt-get install ruby1.9.1 ruby1.9.1-dev rubygems1.9.1 irb1.9.1 ri1.9.1 rdoc1.9.1 g++ ** Bring Ruby into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.8 500 \ --slave /usr/bin/ri ri /usr/bin/ri1.8 \ --slave /usr/bin/irb irb /usr/bin/irb1.8 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.8 ** Bring Ruby 1.9 into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 400 \ --slave /usr/bin/ri ri /usr/bin/ri1.9.1 \ --slave /usr/bin/irb irb /usr/bin/irb1.9.1 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.9.1 ** Choose which Ruby you want, system-wide root@desktop:~# update-alternatives --config ruby :and then choose Ruby 1.9.1 from the menu ** Check it works with root@desktop:~# ruby --version :(Instructions from [http://michalf.me/blog:make-ruby-1-9-default-on-ubuntu-9-10-karmic-koala Michal Frackowiak's blog]) :Or, if you want to install different versions on a per-user basis, or switch quickly between them, use [http://rvm.beginrescueend.com/ RVM]. * Build <code>gem</code> from [http://rubygems.org/pages/download source], to avoid it conflicting with <code>apt</code> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <code>sudo ruby rubygemsx.x.x/setup.rb</code>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :(The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both.) * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install sqlite3-ruby root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install sqlite3-ruby root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install termios ==RVM== If using RVM to keep different Ruby versions around, you need to install the Readline development headers first: root@desktop:~# apt-get install libreadline-dev user@desktop:~$ gem install rails rake rcov rspec gemcutter git jeweler capistrano mysql sqlite3 git shoulda :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] eba33032ff90ea79556c7dbf448f1e4327b4abf4 2359 2358 2011-07-01T10:41:48Z Neil 2 wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient-dev libruby-extras \ libyaml-ruby libzlib-ruby libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@desktop:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient-dev \ libyaml-ruby libzlib-ruby libruby-extras libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * (Optional) Installing Ruby 1.9.2 on Ubuntu, making a system-wide change with the <code>alternatives</code> system. ** Install Ruby 1.9.2 (still in the 1.9.1 package) root@desktop:~# apt-get install ruby1.9.1 ruby1.9.1-dev rubygems1.9.1 irb1.9.1 ri1.9.1 rdoc1.9.1 g++ ** Bring Ruby into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.8 500 \ --slave /usr/bin/ri ri /usr/bin/ri1.8 \ --slave /usr/bin/irb irb /usr/bin/irb1.8 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.8 ** Bring Ruby 1.9 into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 400 \ --slave /usr/bin/ri ri /usr/bin/ri1.9.1 \ --slave /usr/bin/irb irb /usr/bin/irb1.9.1 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.9.1 ** Choose which Ruby you want, system-wide root@desktop:~# update-alternatives --config ruby :and then choose Ruby 1.9.1 from the menu ** Check it works with root@desktop:~# ruby --version :(Instructions from [http://michalf.me/blog:make-ruby-1-9-default-on-ubuntu-9-10-karmic-koala Michal Frackowiak's blog]) :Or, if you want to install different versions on a per-user basis, or switch quickly between them, use [http://rvm.beginrescueend.com/ RVM]. * Build <code>gem</code> from [http://rubygems.org/pages/download source], to avoid it conflicting with <code>apt</code> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <code>sudo ruby rubygemsx.x.x/setup.rb</code>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :(The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both.) * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install sqlite3-ruby root@server:~# gem install passenger root@server:~# gem install mongrel_cluster :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install sqlite3-ruby root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install rspec root@desktop:~# gem install cucumber root@desktop:~# gem install yard root@desktop:~# gem install termios ==RVM== If using RVM to keep different Ruby versions around, you need to install the Readline development headers first: root@desktop:~# apt-get install libreadline-dev user@desktop:~$ gem install rails rake rcov rspec gemcutter git jeweler capistrano mysql sqlite3 shoulda cucumber yard :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 1bfa08bd1c362ddb4f819d00b5b4bc18d8c36d0e 2375 2359 2011-09-23T13:02:44Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * Install the following packages: :Server root@server:~# apt-get install ruby-full build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby ruby-elisp libmysqlclient-dev libruby-extras \ libyaml-ruby libzlib-ruby libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby :Desktop root@desktop:~# apt-get install ruby-full build-essential ruby-elisp libmysqlclient-dev \ libyaml-ruby libzlib-ruby libruby-extras libopenssl-ruby \ sqlite3 libmysql-ruby \ libmysql-ruby libsqlite3-ruby libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev librmagick-ruby * (Optional) Installing Ruby 1.9.2 on Ubuntu, making a system-wide change with the <code>alternatives</code> system. ** Install Ruby 1.9.2 (still in the 1.9.1 package) root@desktop:~# apt-get install ruby1.9.1 ruby1.9.1-dev rubygems1.9.1 irb1.9.1 ri1.9.1 rdoc1.9.1 g++ ** Bring Ruby into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.8 500 \ --slave /usr/bin/ri ri /usr/bin/ri1.8 \ --slave /usr/bin/irb irb /usr/bin/irb1.8 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.8 ** Bring Ruby 1.9 into the <code>alternatives</code> system root@desktop:~# update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 400 \ --slave /usr/bin/ri ri /usr/bin/ri1.9.1 \ --slave /usr/bin/irb irb /usr/bin/irb1.9.1 \ --slave /usr/bin/rdoc rdoc /usr/bin/rdoc1.9.1 ** Choose which Ruby you want, system-wide root@desktop:~# update-alternatives --config ruby :and then choose Ruby 1.9.1 from the menu ** Check it works with root@desktop:~# ruby --version :(Instructions from [http://michalf.me/blog:make-ruby-1-9-default-on-ubuntu-9-10-karmic-koala Michal Frackowiak's blog]) :Or, if you want to install different versions on a per-user basis, or switch quickly between them, use [http://rvm.beginrescueend.com/ RVM]. * Build <code>gem</code> from [http://rubygems.org/pages/download source], to avoid it conflicting with <code>apt</code> root@server:~# tar -xzf rubygems-1.2.0.tgz root@server:~# rm rubygems-1.2.0.tgz root@server:~# cd rubygems-1.2.0 root@server:~# ruby setup.rb root@server:~# cd .. root@server:~# rm -r rubygems-1.2.0 root@server:~# ln -s /usr/bin/gem1.8 /usr/bin/gem :('''Do not''' use <code>sudo ruby rubygemsx.x.x/setup.rb</code>) * Update <code>gem</code> root@server:~# gem update --system root@server:~# gem install rubygems-update root@server:~# update_rubygems :(The first step is needed to upgrade to 1.2.0; the last two steps are needed for the upgrade to 1.3.1. It's best to do them both.) * Install the required gems. On the server, install: root@server:~# gem update root@server:~# gem install rails root@server:~# gem install mysql root@server:~# gem install sqlite3 root@server:~# gem install passenger root@server:~# gem install mongrel_cluster root@server:~# gem install bundler root@server:~# gem install jeweler :On the development machine, install: root@desktop:~# gem update root@desktop:~# gem install rails root@desktop:~# gem install mysql root@desktop:~# gem install sqlite3 root@desktop:~# gem install passenger root@desktop:~# gem install capistrano root@desktop:~# gem install rspec root@desktop:~# gem install cucumber root@desktop:~# gem install yard root@desktop:~# gem install termios root@desktop:~# gem install bundler root@desktop:~# gem install jeweler ==RVM== If using RVM to keep different Ruby versions around, you need to install the Readline development headers first: root@desktop:~# apt-get install libreadline-dev user@desktop:~$ gem install rails rake rcov rspec gemcutter git jeweler capistrano mysql sqlite3 shoulda cucumber yard :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] 57588ca76c597328cb5523928b396c07fb5e9b68 0 1544 2360 2312 2011-07-06T10:51:53Z Neil 2 /* Getting the servers running */ Added note on disabling rndc wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Disable rndc by adding this line to the top of <code>/etc/bind/named.conf</code> controls { }; :I could never get it working, and always use the <code>/etc/init.d/bind9</code> init script to control bind. * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 1.2.3.4 nameserver 1.2.3.5 # nameserver 192.168.1.1 :the last two nameservers are the nameservers of my ISP. * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/named root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run root@server:~# chown -R bind:bind /chroot/bind/var/cache * Modify the file <code>/etc/default/bind9</code> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * '''For Ubuntu 8.04:''' Modify <code>/etc/default/syslogd</code> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * '''For Ubuntu 10.04:''' Create <code>/etc/rsyslog.d/bind-chroot.conf</code> to log messages from the chrooted BIND: $AddUnixListenSocket /chroot/bind/dev/log * Make AppArmor allow BIND access to the chroot jail. Modify <code>/etc/apparmor.d/usr.sbin.named</code> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/lib/bind/ rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/cache/bind/ rw, /chroot/bind/var/run/named/named.pid w, /chroot/bind/var/run/named/session.key w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, system logger, and BIND: root@server:~# /etc/init.d/apparmor restart ** '''For Ubuntu 8.04:''' root@server:~# /etc/init.d/sysklogd restart ** '''For Ubuntu 10.04:''' root@server:~# service rsyslog restart * For both: root@server:~# /etc/init.d/bind9 restart * Check <code>/var/log/syslog</code> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] ab6c22a52b81dca99efed158a6811eedfb664926 2367 2360 2011-08-30T00:08:59Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Disable rndc by adding this line to the top of <code>/etc/bind/named.conf</code> controls { }; :I could never get it working, and always use the <code>/etc/init.d/bind9</code> init script to control bind. * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 192.168.2.251 nameserver 192.168.2.252 :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/named root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run root@server:~# chown -R bind:bind /chroot/bind/var/cache * Modify the file <code>/etc/default/bind9</code> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * '''For Ubuntu 8.04:''' Modify <code>/etc/default/syslogd</code> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * '''For Ubuntu 10.04:''' Create <code>/etc/rsyslog.d/bind-chroot.conf</code> to log messages from the chrooted BIND: $AddUnixListenSocket /chroot/bind/dev/log * Make AppArmor allow BIND access to the chroot jail. Modify <code>/etc/apparmor.d/usr.sbin.named</code> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/lib/bind/ rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/cache/bind/ rw, /chroot/bind/var/run/named/named.pid w, /chroot/bind/var/run/named/session.key w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, * Restart AppArmor, system logger, and BIND: root@server:~# /etc/init.d/apparmor restart ** '''For Ubuntu 8.04:''' root@server:~# /etc/init.d/sysklogd restart ** '''For Ubuntu 10.04:''' root@server:~# service rsyslog restart * For both: root@server:~# /etc/init.d/bind9 restart * Check <code>/var/log/syslog</code> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] cb537f171a2530b93bc940c6fcfb3985059e5337 0 1609 2361 2350 2011-07-23T18:35:45Z Neil 2 moved [[Sous vide recipies]] to [[Sous vide recipes]]:&#32;Spelling! wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag with a tablespoon of duck fat. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton (untried)== No aromatics beyond a little salt and pepper. 70⁰C for 24 hours. From [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery], if you can get the link to work. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] 3c3b2c65211bbdaa6ea7e98d0529281233740863 2363 2361 2011-07-23T18:37:19Z Neil 2 /* Duck leg confit */ wikitext text/x-wiki Some recipies, and links to other recipies. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton (untried)== No aromatics beyond a little salt and pepper. 70⁰C for 24 hours. From [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery], if you can get the link to work. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] bf1355dfd53735dfbfd18872e0ed0fceb5016587 2364 2363 2011-07-24T07:53:10Z Neil 2 wikitext text/x-wiki Some recipes, and links to other recipes. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton (untried)== No aromatics beyond a little salt and pepper. 70⁰C for 24 hours. From [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery], if you can get the link to work. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] f0f0d3561b632a5278d33a7681b07153fb496897 2368 2364 2011-09-04T12:41:08Z Neil 2 /* Mutton (untried) */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] ba472ea413a9d6837872e7ad91a6207352b579f9 2369 2368 2011-09-04T12:42:30Z Neil 2 /* Mutton */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] cb39fcea8a6d3aaea77c4a665e35efb7ee3b0c7c 2370 2369 2011-09-05T19:25:54Z Neil 2 Added pork ribs wikitext text/x-wiki Some recipes, and links to other recipes. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] c709b854b234b04868b07394bfc4bf22526eb501 2403 2370 2011-10-10T18:46:24Z Neil 2 /* Duck leg confit */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] d64c168ea1ae27bc389e5f2993fb1622eff40371 0 1615 2365 2346 2011-07-24T21:02:02Z Neil 2 wikitext text/x-wiki * Freeze liquid marinades before using them. Liquids will just get sucked into the clamp vacuum sealer and damage it. * Double seal all bags (i.e. after the first evacuate-and-seal, create an additional seal at both ends of the bag, inside the original seals. * If you're cooking for more than a few hours, double-bag the food. * Put a handful of glass nuggets (of the type used in crafts and flower arranging) in the bottom of the outer bag before sealing it. That will stop the bags from floating. The buoyancy of the food will keep it floating vertically, allowing better water circulation around the food. However, the glass may degrade and crack, to don't put them in the same bag as the food. * Before cooking, scald the sealed bags of food for 30 seconds in boiling water. That won't cook the food, but will destroy any bacteria on its surface. * Ensure that the food is no more than 10% of the water volume. [[Category:Sous vide]] e465242b8410534d8e32e53f946cc5199d32610b 0 1541 2366 2155 2011-08-30T00:07:02Z Neil 2 /* Hostname */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/resolv.conf</code> to include domain domain.tld search domain.tld :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 * Change <tt>/etc/resolv.conf</tt> to nameserver 192.168.1.1 (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; [[Category:Server setup]] 784574097817c13aafe7e8441c14bf1dac442192 0 1616 2371 2011-09-22T08:35:47Z Neil 2 Created page with 'These are some steps to go though when starting a new Ruby project. ==Create using Jeweler for new gems== If you want to turn your project into a gem, use Jeweler to set it up: …' wikitext text/x-wiki These are some steps to go though when starting a new Ruby project. ==Create using Jeweler for new gems== If you want to turn your project into a gem, use Jeweler to set it up: user@desktop:~$ jeweler my-new-project You then need to modify a few files. * Gemfile: update the <code>rspec</code> line to use RSpec 2.6.x and use RDoc: group :development do gem "rspec", "~> 2.6.0" gem "bundler", "~> 1.0.0" gem "jeweler", "~> 1.6.2" gem "rcov", ">= 0" gem "rdoc" end * Rakefile: update it to use the new version of RDoc (thanks to [http://matthew.mceachen.us/blog/howto-fix-rake-rdoctask-is-deprecated-use-rdoc-task-instead-1169.html Hints and Kinks]). Change the two lines: require 'rake/rdoctask' Rake::RDocTask.new(:rdoc) do |rdoc| :to be require 'rdoc/task' RDoc::Task.new do |rdoc| * gitignore: Make sure it doesn't pick up your editor temp files. Follow the instructions in the default file. ==Set up git== Follow [[Git setup#Creating new repositories in gitosis|the standard instructions]] for setting up a new repository in Gitosis and Gitweb. To echo it to Github as well, do: user@desktop:~/my-new-project$ git remote add origin git@git.domain.tld:my-new-project.git user@desktop:~/my-new-project$ git remote add github git@github.com:GithubUser/my-new-project.git (Jeweler uses Github as the origin. If you want to change that, use <code>git remote rename origin github</code> first.) ==Create directory structure== Ruby files go in <code>my-new-project/lib/module-name/</code>, RSpec files got in <code>my-new-project/spec/module-name</code>. Create <code>my-new-project/lib/module-name.rb</code> to load all the files: require 'module-name/file1' require 'module-name/file2' require 'module-name/file3' ==Set up RSpec== * Create <code>my-new-project/.rspec</code> to hold common options: --color * Create <code>my-new-project/spec/spec_helper.rb</code> $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) $LOAD_PATH.unshift(File.dirname(__FILE__)) require 'rspec' require 'module-name' # Requires supporting files with custom matchers and macros, etc, # in ./support/ and its subdirectories. Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f} RSpec.configure do |config| end ==Use of Rake== * <code>rake spec</code> runs the tests. * <code>rake rdoc</code> creates documentation. [[Category:Ruby]] 04f7daa68aed0d32c07f5f5636d846359ca56699 2372 2371 2011-09-22T08:37:06Z Neil 2 /* Create using Jeweler for new gems */ wikitext text/x-wiki These are some steps to go though when starting a new Ruby project. ==Create using Jeweler for new gems== If you want to turn your project into a gem, use Jeweler to set it up: user@desktop:~$ jeweler --rspec my-new-project You then need to modify a few files. * Gemfile: update the <code>rspec</code> line to use RSpec 2.6.x and use RDoc: group :development do gem "rspec", "~> 2.6.0" gem "bundler", "~> 1.0.0" gem "jeweler", "~> 1.6.2" gem "rcov", ">= 0" gem "rdoc" end * Rakefile: update it to use the new version of RDoc (thanks to [http://matthew.mceachen.us/blog/howto-fix-rake-rdoctask-is-deprecated-use-rdoc-task-instead-1169.html Hints and Kinks]). Change the two lines: require 'rake/rdoctask' Rake::RDocTask.new(:rdoc) do |rdoc| :to be require 'rdoc/task' RDoc::Task.new do |rdoc| * gitignore: Make sure it doesn't pick up your editor temp files. Follow the instructions in the default file. ==Set up git== Follow [[Git setup#Creating new repositories in gitosis|the standard instructions]] for setting up a new repository in Gitosis and Gitweb. To echo it to Github as well, do: user@desktop:~/my-new-project$ git remote add origin git@git.domain.tld:my-new-project.git user@desktop:~/my-new-project$ git remote add github git@github.com:GithubUser/my-new-project.git (Jeweler uses Github as the origin. If you want to change that, use <code>git remote rename origin github</code> first.) ==Create directory structure== Ruby files go in <code>my-new-project/lib/module-name/</code>, RSpec files got in <code>my-new-project/spec/module-name</code>. Create <code>my-new-project/lib/module-name.rb</code> to load all the files: require 'module-name/file1' require 'module-name/file2' require 'module-name/file3' ==Set up RSpec== * Create <code>my-new-project/.rspec</code> to hold common options: --color * Create <code>my-new-project/spec/spec_helper.rb</code> $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) $LOAD_PATH.unshift(File.dirname(__FILE__)) require 'rspec' require 'module-name' # Requires supporting files with custom matchers and macros, etc, # in ./support/ and its subdirectories. Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f} RSpec.configure do |config| end ==Use of Rake== * <code>rake spec</code> runs the tests. * <code>rake rdoc</code> creates documentation. [[Category:Ruby]] 77d183365b8c135d0e3f97783c64d5ec833fe266 2373 2372 2011-09-22T08:41:04Z Neil 2 /* Set up git */ wikitext text/x-wiki These are some steps to go though when starting a new Ruby project. ==Create using Jeweler for new gems== If you want to turn your project into a gem, use Jeweler to set it up: user@desktop:~$ jeweler --rspec my-new-project You then need to modify a few files. * Gemfile: update the <code>rspec</code> line to use RSpec 2.6.x and use RDoc: group :development do gem "rspec", "~> 2.6.0" gem "bundler", "~> 1.0.0" gem "jeweler", "~> 1.6.2" gem "rcov", ">= 0" gem "rdoc" end * Rakefile: update it to use the new version of RDoc (thanks to [http://matthew.mceachen.us/blog/howto-fix-rake-rdoctask-is-deprecated-use-rdoc-task-instead-1169.html Hints and Kinks]). Change the two lines: require 'rake/rdoctask' Rake::RDocTask.new(:rdoc) do |rdoc| :to be require 'rdoc/task' RDoc::Task.new do |rdoc| * gitignore: Make sure it doesn't pick up your editor temp files. Follow the instructions in the default file. ==Set up git== Follow [[Git setup#Creating new repositories in gitosis|the standard instructions]] for setting up a new repository in Gitosis and Gitweb. To echo it to Github as well, do: user@desktop:~/my-new-project$ git remote add origin git@git.domain.tld:my-new-project.git user@desktop:~/my-new-project$ git remote add github git@github.com:GithubUser/my-new-project.git (Jeweler uses Github as the origin. If you want to change that, use <code>git remote rename origin github</code> first.) In use, use: user@desktop:~/my-new-project$ git commit -a -m 'Informative message' user@desktop:~/my-new-project$ git push -u origin master user@desktop:~/my-new-project$ git push -u github master to commit and push the repository. ==Create directory structure== Ruby files go in <code>my-new-project/lib/module-name/</code>, RSpec files got in <code>my-new-project/spec/module-name</code>. Create <code>my-new-project/lib/module-name.rb</code> to load all the files: require 'module-name/file1' require 'module-name/file2' require 'module-name/file3' ==Set up RSpec== * Create <code>my-new-project/.rspec</code> to hold common options: --color * Create <code>my-new-project/spec/spec_helper.rb</code> $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) $LOAD_PATH.unshift(File.dirname(__FILE__)) require 'rspec' require 'module-name' # Requires supporting files with custom matchers and macros, etc, # in ./support/ and its subdirectories. Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f} RSpec.configure do |config| end ==Use of Rake== * <code>rake spec</code> runs the tests. * <code>rake rdoc</code> creates documentation. [[Category:Ruby]] 45b6b9f0a75204ceee63780c0c627d6d4b2cbc87 2374 2373 2011-09-22T08:42:04Z Neil 2 /* Create using Jeweler for new gems */ wikitext text/x-wiki These are some steps to go though when starting a new Ruby project. ==Create using Jeweler for new gems== If you want to turn your project into a gem, use Jeweler to set it up: user@desktop:~$ jeweler --rspec my-new-project You then need to modify a few files. * Gemfile: update the <code>rspec</code> line to use RSpec 2.6.x and use RDoc: group :development do gem "rspec", "~> 2.6.0" gem "bundler", "~> 1.0.0" gem "jeweler", "~> 1.6.2" gem "rcov", ">= 0" gem "rdoc" end * Rakefile: update it to use the new version of RDoc (thanks to [http://matthew.mceachen.us/blog/howto-fix-rake-rdoctask-is-deprecated-use-rdoc-task-instead-1169.html Hints and Kinks]). Change the two lines: require 'rake/rdoctask' Rake::RDocTask.new(:rdoc) do |rdoc| :to be require 'rdoc/task' RDoc::Task.new do |rdoc| * Rakefile: update the gem description in the <code>gem.summary</code> and <code>gem.description</code> lines. * gitignore: Make sure it doesn't pick up your editor temp files. Follow the instructions in the default file. ==Set up git== Follow [[Git setup#Creating new repositories in gitosis|the standard instructions]] for setting up a new repository in Gitosis and Gitweb. To echo it to Github as well, do: user@desktop:~/my-new-project$ git remote add origin git@git.domain.tld:my-new-project.git user@desktop:~/my-new-project$ git remote add github git@github.com:GithubUser/my-new-project.git (Jeweler uses Github as the origin. If you want to change that, use <code>git remote rename origin github</code> first.) In use, use: user@desktop:~/my-new-project$ git commit -a -m 'Informative message' user@desktop:~/my-new-project$ git push -u origin master user@desktop:~/my-new-project$ git push -u github master to commit and push the repository. ==Create directory structure== Ruby files go in <code>my-new-project/lib/module-name/</code>, RSpec files got in <code>my-new-project/spec/module-name</code>. Create <code>my-new-project/lib/module-name.rb</code> to load all the files: require 'module-name/file1' require 'module-name/file2' require 'module-name/file3' ==Set up RSpec== * Create <code>my-new-project/.rspec</code> to hold common options: --color * Create <code>my-new-project/spec/spec_helper.rb</code> $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) $LOAD_PATH.unshift(File.dirname(__FILE__)) require 'rspec' require 'module-name' # Requires supporting files with custom matchers and macros, etc, # in ./support/ and its subdirectories. Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f} RSpec.configure do |config| end ==Use of Rake== * <code>rake spec</code> runs the tests. * <code>rake rdoc</code> creates documentation. [[Category:Ruby]] 632e4233b008a8b4c7d78c89032cb14ab6739332 2376 2374 2011-09-23T13:15:49Z Neil 2 /* Create using Jeweler for new gems */ wikitext text/x-wiki These are some steps to go though when starting a new Ruby project. ==Create using Jeweler for new gems== * If you want to turn your project into a gem, use Jeweler to set it up: user@desktop:~$ jeweler --rspec my-new-project user@desktop:~$ cd my-new-project * Create the version number for release: user@desktop:~/my-new-project$ rake version:write MAJOR=0 MINOR=1 PATCH=0 You then need to modify a few files. * Gemfile: update the <code>rspec</code> line to use RSpec 2.6.x and use RDoc: group :development do gem "rspec", "~> 2.6.0" gem "bundler", "~> 1.0.0" gem "jeweler", "~> 1.6.2" gem "rcov", ">= 0" gem "rdoc" end * Rakefile: update it to use the new version of RDoc (thanks to [http://matthew.mceachen.us/blog/howto-fix-rake-rdoctask-is-deprecated-use-rdoc-task-instead-1169.html Hints and Kinks]). Change the two lines: require 'rake/rdoctask' Rake::RDocTask.new(:rdoc) do |rdoc| :to be require 'rdoc/task' RDoc::Task.new do |rdoc| * Rakefile: update the gem description in the <code>gem.summary</code> and <code>gem.description</code> lines. * gitignore: Make sure it doesn't pick up your editor temp files. Follow the instructions in the default file. ==Set up git== Follow [[Git setup#Creating new repositories in gitosis|the standard instructions]] for setting up a new repository in Gitosis and Gitweb. To echo it to Github as well, do: user@desktop:~/my-new-project$ git remote add origin git@git.domain.tld:my-new-project.git user@desktop:~/my-new-project$ git remote add github git@github.com:GithubUser/my-new-project.git (Jeweler uses Github as the origin. If you want to change that, use <code>git remote rename origin github</code> first.) In use, use: user@desktop:~/my-new-project$ git commit -a -m 'Informative message' user@desktop:~/my-new-project$ git push -u origin master user@desktop:~/my-new-project$ git push -u github master to commit and push the repository. ==Create directory structure== Ruby files go in <code>my-new-project/lib/module-name/</code>, RSpec files got in <code>my-new-project/spec/module-name</code>. Create <code>my-new-project/lib/module-name.rb</code> to load all the files: require 'module-name/file1' require 'module-name/file2' require 'module-name/file3' ==Set up RSpec== * Create <code>my-new-project/.rspec</code> to hold common options: --color * Create <code>my-new-project/spec/spec_helper.rb</code> $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) $LOAD_PATH.unshift(File.dirname(__FILE__)) require 'rspec' require 'module-name' # Requires supporting files with custom matchers and macros, etc, # in ./support/ and its subdirectories. Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f} RSpec.configure do |config| end ==Use of Rake== * <code>rake spec</code> runs the tests. * <code>rake rdoc</code> creates documentation. [[Category:Ruby]] f81f85ca75674607cf5af7cc216fb8ab99aad8b6 0 1604 2377 2341 2011-09-25T09:39:25Z Neil 2 moved [[Simple sous vide setup]] to [[Sous vide setup mk 1]] wikitext text/x-wiki Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ==Equipment== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket (under £20.00). Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's not too expensive (£26.00) and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay (£5.00). * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket (about £3.00). * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. The controller also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ==Wiring== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ==Controller tuning== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 327161870ccdeea42df885342b2202f35e52bc3a 2385 2377 2011-09-25T09:52:58Z Neil 2 Added link to mk 3 cooker wikitext text/x-wiki Professional sous kits can cost hundreds, if not thousands of pounds. But it's relatively easy to get started with cheap kit. However, the instructions for how to do it are sparse and confusing. Hence this page. This is a very simple sous vide setup, inspired by Jeff Potter's [http://www.cookingforgeeks.com/blog/posts/diy-sous-vide/ DIY sous vide] description. It uses a slow cooker as the food container and heat source and a temperature controller with a thermocouple to keep things at a stable temperature. ==Equipment== To do sous vide cooking at home, you will need: * A '''slow cooker'''. I got a cheap one on sale at the supermarket (under £20.00). Note that it has to be a cheap one, with a mechanical switch, as you'll be switching the power to it. Expensive, digital ones are liable to get confused. * A '''temperature controller'''. The one suggested by Jeff Potter isn't available, so I got a TET 612 controller from eBay. It's probably overkill for this job, but it's not too expensive (£26.00) and will do for the more complex setup below if you want to go that far. (I wasn't able to find a cheaper, lower spec alternative.) * A '''thermocouple'''. A type J thermocouple for monitoring the temperature. Again, cheap on eBay (£5.00). * An '''extension lead'''. Buying a four-way extension socket was the cheapest way of getting an electrical plug and socket (about £3.00). * An old '''plastic food containter'''. You'll have one lying around. It's to keep the controller neat and tidy. * '''Insulation tape''' and '''electrical connector'''. Needed to keep the electrics safe. * Microwave-safe, sealable '''food bags'''. To put the food in. Make sure they're watertight. The TET 612 has an internal relay, capable of delivering a current of 3A. That's easily sufficient for the 210W slow cooker I have. It means I don't need a separate relay. Check that your slow cooker doesn't draw more than 720W: if it does, you'll need a separate relay. The controller also runs of a 240V mains supply, so no need for a separate power supply. The bad news is that the manual supplied with the controller is very badly translated technical chinese. Luckily, some [http://www.homebrewtalk.com/f51/tet612-pid-133807/ US homebrewers have the same problem] can have found an [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 equivalent controller] with a [http://auberins.com/images/Manual/SYL-2362%20instruction%201.6.pdf better manual]. Based on that, these are the instructions to use. ==Wiring== [[Image:Tet612-sous-vide-wiring.png|right|thumb|250px|Wiring layout for the simple sous vide controller. Numbered circles are the terminal numbers.]] The idea is to use the relay in the temperature controller to switch power to the slow cooker. It will do so in a way to keep the slow cooker at the desired temperature. You can see the principle from the wiring diagram on the right. By using an extension lead, there's no need to cut the power cord for the slow cooker. The relay outputs are noted as 'Alarm 2.' # Cut out a 30cm section of lead in the extension cord. The section should come out from about in the middle. Remove the outer insulation from this section, separate the wires, and remove about 2cm of insulation from each end of the live (brown) wire. Discard the neutral and earth wires from this section. # Remove the outer insulation from about 5cm of each cut end of the extension lead cables and strip about 2cm of inner insulation from each wire. # Twist together the live wire from the plug end and one end of the brown connecting wire. Attach these wires to terminal 1 of the controller. # Attach the other end of the connecting wire to terminal 13 of the controller. # Attach the live wire from the socket cable to terminal 14 of the controller. # Twist together the neutral wires from the plug and socket cables. Connect them to terminal 2 of the controller. # Use the electrical connector to join the two earth leads. # Connect the thermocouple: attach the positive (red) wire to terminal 9, the other lead to terminal 10. # Cover all the contacts and connections with insulation tape to stop accidental shorts. You're now ready to set up the controller. Plug it in and the lights should come on! The top numbers on the display should be the current temperature, read by the thermocouple. Use a cup of hot water and a cup of cold water to check that the thermocouple responds to temperature changes. Plug a small lamp into the extension lead to check the relay is working. Use the 'Up' and 'Down' arrow buttons to adjust the target temperature. Once everything's working, place the controller in the plastic food box. Cut a chunk out of the rear of the box to run the cables through. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 1 || Use the internal relay for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ==Controller tuning== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. Do it with the slow cooker connected and filled with water that is just below the target temperature. The controller will move the water temperature above and below the target three times, so it could take a few hours to do. Trust the parameters the controller produces. The only one to check is the 'ot' parameter, the output time. This controls how often the relay switches. It should be 20 (seconds) or more, to prevent the relay wearing out too quickly. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 00.4 |- | I || 1011 |- | d || 252 |- | Souf || 0.2 |- | ot || 040 |- | Filt || 0 |} ==Moving on== This setup is easy to get going, but has its limitations. The main one is the small capacity of the slow cooker. You need plenty of water around the food to allow water circulation and to ensure everything is at an even temperature. That means you can only cook meat sufficient for 2-3 people. If you want more capacity, you'll need to produce are more sophisticated cooker, such as my [[sous vide setup mk 3]]. (The [[sous vide setup mk 2|mk 2]] was a learning experience, not to be repeated.) ==See also== * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller]. * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612). [[Category:Sous vide]] 80520bab0e9502ee11a24b46ba8ebf92db084060 0 1617 2378 2011-09-25T09:39:25Z Neil 2 moved [[Simple sous vide setup]] to [[Sous vide setup mk 1]] wikitext text/x-wiki #REDIRECT [[Sous vide setup mk 1]] 7d6261ab44f8f4e042cd366c4f47a7c7d7abe704 0 1618 2380 2011-09-25T09:39:46Z Neil 2 moved [[Advanced sous vide setup]] to [[Sous vide setup mk 2]] wikitext text/x-wiki #REDIRECT [[Sous vide setup mk 2]] a83bad45cfa93ef1634344121f470b2194107bbb 0 1 2382 2307 2011-09-25T09:40:54Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. fe7cd84012a23730cc3246549ca4d6fb636ddb4f 0 1619 2384 2011-09-25T09:52:37Z Neil 2 Created page with 'The [[sous vide setup mk 2]] had some problems which meant it wasn't a long-term solution. The problems were: # aquarium motors are only rated up to about 35⁰C, and stop workin…' wikitext text/x-wiki The [[sous vide setup mk 2]] had some problems which meant it wasn't a long-term solution. The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-underneath.jpg|200px|thumb|right|The underside of the cooker, showing the arrangement of the parts that go into the water]] * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': Most standard aquarium power heads are only rated up to 35⁰C and quickly fail at sous vide temperatures of 60⁰C and above. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** [http://cgi.ebay.com/DC-6V-Mini-Submersible-Water-Pump-35-GPH-P-32A-/260688654675?pt=LH_DefaultDomain_0&hash=item3cb2413153#ht_3239wt_1141 DC 6V Mini Submersible Water Pump 35 GPH (P-32A)], a submersible pump rated up to 80⁰C ** [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 DC 12V Electric Submersible Water Pump 95 GPH (P-38B)], a submersible pump rated up to 80⁰C ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Perspex/acrylic sheets''': These are very large and expensive. You can probably get a deal from your local DIY shop on a damaged sheet as you'll only need a tiny amount. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. ==Construction tips== [[File:Sous-vide-cooker-inside.jpg|200px|thumb|right|Inside the box, showing a rats' nest of wires]] * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Perspex softens quite nicely over a gas ring''': You won't need to sacrifice a pan to protect the perspex, so long as you're careful and don't hold the perspex too close to the flame. * '''Don't screw the pump to its support''': I'm very wary of driving metal screws into the sealed housing of a motor that operates at 240V and will be immersed in water. Luckily, hot glue is sufficient to hold the pump in place. (But not the bracket itself: that still needs bolts.) * '''Perspex clamps are for small boxes''' The combination of larger box, heavier heaters, and the thinner perspex I was able to obtain conspired to make me reject the C-clamp style of attachment for the cooker as described in Make. As I'll only be using the same pot all the time, it was easier to cut a hole in the water-bath lid and place the cooker over that. The lid supports the box while the heaters, etc. dangle in the water. * '''Sandpaper surfaces, especially perspex, before gluing''': Perspex is very smooth and glue can't key into the surface. A bit of sandpapering means that things don't fall apart instantly. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-advanced.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==Extensions== The C-clamp style attachment is a more convenient way of securing the heater, as well as allowing it to be used on different pots. But, as I said above, I'w wary of using perpex to hold the cooker stable, as its large, heavy, and perspex weakens in higher temperatures. I'm investigating using a piece of aluminium as a clamp instead. ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. * Don't run the cooker above 75⁰C. I once tried cooking carrots at 85⁰C and melted the pump and the perspex pump support. The pump never worked again. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 36fcd7630b2ce58d291c88f59d5007752f163c41 2386 2384 2011-09-25T10:09:31Z Neil 2 wikitext text/x-wiki The [[sous vide setup mk 2]] had some problems which meant it wasn't a long-term solution. The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 9db8ee5ed6a0a30bd4438a6decf3cee50022f2e7 2388 2386 2011-09-25T10:27:37Z Neil 2 /* Wiring */ wikitext text/x-wiki The [[sous vide setup mk 2]] had some problems which meant it wasn't a long-term solution. The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 03ac520354894640ce42f6159835014618903ba8 2390 2388 2011-09-25T10:37:05Z Neil 2 wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 2]] had some problems which meant it wasn't a long-term solution. The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 726d67f9ef6bef054061820ec67bf8eb8f513e67 2391 2390 2011-09-25T10:39:07Z Neil 2 /* Materials */ wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 2]] had some problems which meant it wasn't a long-term solution. The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. Alternatives are: ** [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming ** [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming ** [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this ** an aquarium air pump, without an airstone, with the bubbles providing circulation :All the pumps require an additional power supply. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction== # Ensure you have sufficient room on the base of your box for all the components. You'll need the three heaters inline, with the pump outflow pushing water along the centre of the heater coils. The thermocouple should go adjacent to the pump, upstream of the heaters. You'll also need room for the clamp. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 6874f16b080980cb0b5fab489af02f12b43bc921 2392 2391 2011-09-25T10:41:34Z Neil 2 wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 2]] had some problems which meant it wasn't a long-term solution. The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. Here, I give some hints and tips on how I made my cooker. You'll need to read the original project description for these variations to make sense. ==Parts== * '''Heaters''': The main problem I had was getting the immersion heaters. There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones that look capable of reaching the required temperatures. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction== # Ensure you have sufficient room on the base of your box for all the components. You'll need the three heaters inline, with the pump outflow pushing water along the centre of the heater coils. The thermocouple should go adjacent to the pump, upstream of the heaters. You'll also need room for the clamp. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==Alternative pumps== The hardest part to sources is a pump that is capable of handling water at 80⁰C+ but still small and cheap enough to be viable the cooker. The P38-B is the best I could find. Alternatives are: * [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming * [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming * [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this * an aquarium air pump, without an airstone, with the bubbles providing circulation All the pumps require an additional power supply. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] a163349ae1528faa03a4a608331f1ec25635ee94 2393 2392 2011-09-25T10:47:35Z Neil 2 wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 1]] is fine for getting started, but doesn't have sufficient capacity for most meals. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. In addition, it has some problems which meant it wasn't a long-term solution (see the [[sous vide setup mk 2]] for learning about it the hard way). The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Although this description is fairly self-contained, you'll need to read the [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 original project description] for these variations to make sense. ==Parts== * '''Heaters''': There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones with a decent power rating. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. :It also takes up a lot of room. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction== # Ensure you have sufficient room on the base of your box for all the components. You'll need the three heaters inline, with the pump outflow pushing water along the centre of the heater coils. The thermocouple should go adjacent to the pump, upstream of the heaters. You'll also need room for the clamp. # Space inside the box is tight, too. Make sure you know where all the components are going. I had to saw off the corner of the base of the relay's heat sink to allow the pump power lead into the box. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==Alternative pumps== The hardest part to sources is a pump that is capable of handling water at 80⁰C+ but still small and cheap enough to be viable the cooker. The P38-B is the best I could find. Alternatives are: * [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming * [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming * [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this * an aquarium air pump, without an airstone, with the bubbles providing circulation All the pumps require an additional power supply. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 0116b37a9883928bf8697bd24a1a37e865386118 2398 2393 2011-09-30T20:07:18Z Neil 2 wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 1]] is fine for getting started, but doesn't have sufficient capacity for most meals. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. In addition, it has some problems which meant it wasn't a long-term solution (see the [[sous vide setup mk 2]] for learning about it the hard way). The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Although this description is fairly self-contained, you'll need to read the [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 original project description] for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-mk3-underside.jpg|200px|thumb|right|The underside of the cooker, showing how the components are arranged]] * '''Heaters''': There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones with a decent power rating. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. :It also takes up a lot of room. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction== [[File:Sous-vide-cooker-mk3-inside.jpg|200px|thumb|right|The arrangement of parts inside the box]] # Ensure you have sufficient room on the base of your box for all the components. You'll need the three heaters inline, with the pump outflow pushing water along the centre of the heater coils. The thermocouple should go adjacent to the pump, upstream of the heaters. You'll also need room for the clamp. # Space inside the box is tight, too. Make sure you know where all the components are going. I had to saw off the corner of the base of the relay's heat sink to allow the pump power lead into the box. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==Alternative pumps== The hardest part to sources is a pump that is capable of handling water at 80⁰C+ but still small and cheap enough to be viable the cooker. The P38-B is the best I could find. Alternatives are: * [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming * [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming * [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this * an aquarium air pump, without an airstone, with the bubbles providing circulation All the pumps require an additional power supply. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 8814fbc7c2a87e7ab249beeffa9dec1d4a14e7cb 2399 2398 2011-10-03T09:01:23Z Neil 2 /* Construction */ wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 1]] is fine for getting started, but doesn't have sufficient capacity for most meals. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. In addition, it has some problems which meant it wasn't a long-term solution (see the [[sous vide setup mk 2]] for learning about it the hard way). The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Although this description is fairly self-contained, you'll need to read the [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 original project description] for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-mk3-underside.jpg|200px|thumb|right|The underside of the cooker, showing how the components are arranged]] * '''Heaters''': There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones with a decent power rating. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. :It also takes up a lot of room. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction== [[File:Sous-vide-cooker-mk3-inside.jpg|200px|thumb|right|The arrangement of parts inside the box]] # Ensure you have sufficient room on the base of your box for all the components. You'll need the three heaters inline, with the pump outflow pushing water along the centre of the heater coils. The thermocouple should go adjacent to the pump, upstream of the heaters. You'll also need room for the clamp. # Space inside the box is tight, too. Make sure you know where all the components are going. I had to saw off the corner of the base of the relay's heat sink to allow the pump power lead into the box. The 12V power supply has to go on the opposite side of the box from the heaters, which limits places where the power cable can come in from outside. Also, you probably won't be able to mount the controller directly above the heaters, due to lack of vertical space. # Drill the holes in the side of the mounting bracket for the mounting bolts. Use two-part epoxy glue to fix the nuts on the inside of the bracket. # Start by laying out the heaters and pump on the base of the box. Place the heaters as close to the edge as you can while still being able to put glue all round them inside the box. You'll also need to leave room for the mounting bracket. # Make the holes for the heaters and insert them without gluing. Then fix the position of the pump and mounting bracket. You want the pump outflow to be directed through the middle of all three heater coils. Ensure that the pump bracket doesn't overlap the line of where the mounting bracket will be, so that the mounting bracket is the only part that touches the side of the tank. # Make the hole for the thermocouple. # Make holes for the power lead and controller. You may need to trim the lip around the lid of the box where the controller is. # Glue the heaters in place, and screw in the pump bracket, pump, and mounting bracket. I added home-made zinc washers to all the bolts, as sacrificial anodes. # Do all the wiring inside before fixing any components. It'll turn into a bit of a rats nest, but that can't be helped. # Attach the SSR to its heat sink. # Use two-part epoxy glue to secure the SSR heat sink and 12V power supply in place. # The controller stays in place by friction, especially when the lid is secured. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==Alternative pumps== The hardest part to sources is a pump that is capable of handling water at 80⁰C+ but still small and cheap enough to be viable the cooker. The P38-B is the best I could find. Alternatives are: * [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming * [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming * [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this * an aquarium air pump, without an airstone, with the bubbles providing circulation All the pumps require an additional power supply. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://www.freshmealssolutions.com/downloads/PID1500Dtuningguiderelease1.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 8ec50264216db8d7b2fa39b3ad392f213a77bf53 2400 2399 2011-10-03T09:06:04Z Neil 2 /* See also */ wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 1]] is fine for getting started, but doesn't have sufficient capacity for most meals. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. In addition, it has some problems which meant it wasn't a long-term solution (see the [[sous vide setup mk 2]] for learning about it the hard way). The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Although this description is fairly self-contained, you'll need to read the [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 original project description] for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-mk3-underside.jpg|200px|thumb|right|The underside of the cooker, showing how the components are arranged]] * '''Heaters''': There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones with a decent power rating. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. :It also takes up a lot of room. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction== [[File:Sous-vide-cooker-mk3-inside.jpg|200px|thumb|right|The arrangement of parts inside the box]] # Ensure you have sufficient room on the base of your box for all the components. You'll need the three heaters inline, with the pump outflow pushing water along the centre of the heater coils. The thermocouple should go adjacent to the pump, upstream of the heaters. You'll also need room for the clamp. # Space inside the box is tight, too. Make sure you know where all the components are going. I had to saw off the corner of the base of the relay's heat sink to allow the pump power lead into the box. The 12V power supply has to go on the opposite side of the box from the heaters, which limits places where the power cable can come in from outside. Also, you probably won't be able to mount the controller directly above the heaters, due to lack of vertical space. # Drill the holes in the side of the mounting bracket for the mounting bolts. Use two-part epoxy glue to fix the nuts on the inside of the bracket. # Start by laying out the heaters and pump on the base of the box. Place the heaters as close to the edge as you can while still being able to put glue all round them inside the box. You'll also need to leave room for the mounting bracket. # Make the holes for the heaters and insert them without gluing. Then fix the position of the pump and mounting bracket. You want the pump outflow to be directed through the middle of all three heater coils. Ensure that the pump bracket doesn't overlap the line of where the mounting bracket will be, so that the mounting bracket is the only part that touches the side of the tank. # Make the hole for the thermocouple. # Make holes for the power lead and controller. You may need to trim the lip around the lid of the box where the controller is. # Glue the heaters in place, and screw in the pump bracket, pump, and mounting bracket. I added home-made zinc washers to all the bolts, as sacrificial anodes. # Do all the wiring inside before fixing any components. It'll turn into a bit of a rats nest, but that can't be helped. # Attach the SSR to its heat sink. # Use two-part epoxy glue to secure the SSR heat sink and 12V power supply in place. # The controller stays in place by friction, especially when the lid is secured. ==Construction tips== * '''Don't make the holes for the heaters too near the sides of the box''': I couldn't make the holes sufficiently accurate for the heaters to stay in by friction alone, so I had to use hot glue to hold them in place. If you do that, remember to leave sufficient space to get a glue gun round all sides of the heater inside the box. * '''Glue down the relay''': The SSR is quite large and heavy. Once you're sure everything's working, fix the relay in place with a ''small'' dab of hot glue to stop it rattling around when the box is moved. * '''Secure the controller''': Because the controller sits in a slot cut in the side of the box, with the lid on top, the controller is only held on three sides. This means it can tilt slightly, a purely cosmetic concern. To fix it, put a large blob of hot glue on the inside of the lid to press against the controller (you may need more than one layer). Make the blob slightly higher than you need. Place two layers of tinfoil over the glue blob and rub over them the clean tip of the glue gun (without applying glue) until the glue blob is flat and the correct height. Remove the top layer of foil and you will ensure that no glue touches the controller, making it easy to remove the lid if needed. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==Alternative pumps== The hardest part to sources is a pump that is capable of handling water at 80⁰C+ but still small and cheap enough to be viable the cooker. The P38-B is the best I could find. Alternatives are: * [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming * [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming * [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this * an aquarium air pump, without an airstone, with the bubbles providing circulation All the pumps require an additional power supply. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://freshmealssolutions.com/downloads/PID-tuning-guide_R2_V006.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] ff0ecdb0fb68666072f550c55c06c4aa43bfa9c5 2401 2400 2011-10-04T15:26:35Z Neil 2 wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 1]] is fine for getting started, but doesn't have sufficient capacity for most meals. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. In addition, it has some problems which meant it wasn't a long-term solution (see the [[sous vide setup mk 2]] for learning about it the hard way). The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Although this description is fairly self-contained, you'll need to read the [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 original project description] for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-mk3-underside.jpg|200px|thumb|right|The underside of the cooker, showing how the components are arranged]] * '''Heaters''': There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones with a decent power rating. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. :It also takes up a lot of room. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction== [[File:Sous-vide-cooker-mk3-inside.jpg|200px|thumb|right|The arrangement of parts inside the box]] # Ensure you have sufficient room on the base of your box for all the components. You'll need the three heaters inline, with the pump outflow pushing water along the centre of the heater coils. The thermocouple should go adjacent to the pump, upstream of the heaters. You'll also need room for the clamp. # Space inside the box is tight, too. Make sure you know where all the components are going. I had to saw off the corner of the base of the relay's heat sink to allow the pump power lead into the box. The 12V power supply has to go on the opposite side of the box from the heaters, which limits places where the power cable can come in from outside. Also, you probably won't be able to mount the controller directly above the heaters, due to lack of vertical space. # Drill the holes in the side of the mounting bracket for the mounting bolts. Use two-part epoxy glue to fix the nuts on the inside of the bracket. # Start by laying out the heaters and pump on the base of the box. Place the heaters as close to the edge as you can while still being able to put glue all round them inside the box. You'll also need to leave room for the mounting bracket. # Make the holes for the heaters and insert them without gluing. Then fix the position of the pump and mounting bracket. You want the pump outflow to be directed through the middle of all three heater coils. Ensure that the pump bracket doesn't overlap the line of where the mounting bracket will be, so that the mounting bracket is the only part that touches the side of the tank. # Make the hole for the thermocouple. # Make holes for the power lead and controller. You may need to trim the lip around the lid of the box where the controller is. # Glue the heaters in place, and screw in the pump bracket, pump, and mounting bracket. I added home-made zinc washers to all the bolts, as sacrificial anodes. # Do all the wiring inside before fixing any components. It'll turn into a bit of a rats nest, but that can't be helped. # Attach the SSR to its heat sink. # Use two-part epoxy glue to secure the SSR heat sink and 12V power supply in place. # The controller stays in place by friction, especially when the lid is secured. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. ==Alternative pumps== The hardest part to sources is a pump that is capable of handling water at 80⁰C+ but still small and cheap enough to be viable the cooker. The P38-B is the best I could find. Alternatives are: * [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming * [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming * [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this * an aquarium air pump, without an airstone, with the bubbles providing circulation All the pumps require an additional power supply. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://freshmealssolutions.com/downloads/PID-tuning-guide_R2_V006.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 3b427d908a4b5687bec0d628cf257e6cb8c6a592 2402 2401 2011-10-04T15:27:09Z Neil 2 /* In use */ wikitext text/x-wiki [[File:Sous-vide-cooker-mk3.jpg|200px|thumb|right|The cooker in situ above a water bath]] The [[sous vide setup mk 1]] is fine for getting started, but doesn't have sufficient capacity for most meals. Make magazine has a [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 good description of how to make such a cooker]. However, not all of the parts are easily available in the UK, so some modification was needed. In addition, it has some problems which meant it wasn't a long-term solution (see the [[sous vide setup mk 2]] for learning about it the hard way). The problems were: # aquarium motors are only rated up to about 35⁰C, and stop working at sous vide cooking temperatures # perspex softens in heat. so motor mounts and supports will fail at sous vide temperatures # the Fotek SSR requires a heat sink for when its drawing current # the geometry of having the box sticking out into the water bath wastes a lot of space The general principle was fine, but it needed some modification. The cooker unit contains heating elements, a thermocouple, the temperature controller, and an aquarium pump (to mix the water). The whole lot is in a single box, ready to be placed on the edge of any convenient water bath. The alternative is to have two separate boxes, one containing the controller and relay, the other housing the heaters and thermocouple. Two boxes means having less to mount over the water bath, but at the expense of having many leads between the two (swiched mains for the heaters, 12V for the pump, and the thermocouple lead.) Although this description is fairly self-contained, you'll need to read the [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 original project description] for these variations to make sense. ==Parts== [[File:Sous-vide-cooker-mk3-underside.jpg|200px|thumb|right|The underside of the cooker, showing how the components are arranged]] * '''Heaters''': There are plenty of 110V in-cup immersion heaters available on eBay, but very few 240V ones with a decent power rating. I ended up getting some Russian ones from an eBay seller in Ukraine. The plus side is that they're 500W heaters rather than the 300W ones in the original article. That makes the cooker capable of heating quite large water volumes (30l - 40l). The down side is that they're quite a lot larger than the baseline version, so that makes the whole unit much bigger. * '''Enclosure''': Make sure you get one that's big enough to hold the controller and the heaters. I ended up getting a monster box with dimensions 21cm × 13cm × 9cm. It needs to be that big to fit the 16cm required of the three heaters and pump in a line. * '''Controller''': I reused the TET 612 from the [[Sous vide setup mk 1]]. * '''Temperature sensor''': Again, I reused the type J thermocouple from the [[Sous vide setup mk 1]] rather than the PT100 platinum temperature-dependent resistor from Make. * '''Relay''': The easiest solid-state relay (SSR) for me to get hold of (Fotek SSR 25-DA) has screw-down terminal blocks, so no need for any soldering. It's also bigger than the one in Make, but I had plenty of room in my monster box. The downside is that it needs a heatsink block, which takes up a lot of the room in the monster box... * '''Circulation pump''': a [http://cgi.ebay.com/DC-12V-Electric-Submersible-Water-Pump-95-GPH-P-38B-/250722722452?pt=LH_DefaultDomain_0&hash=item3a603d2294#ht_3322wt_1141 P-38B 12V DC Submersible Water Pump], a submersible pump rated up to 80⁰C that pushes out 7.2l/min (95 gal/hr), allowing the heater to use water baths up to around 35l. * A power supply for the pump. I used a replacement LCD monitor power supply. The 5A it can supply is vastly more than the pump needs, but that power supply is ** easily available, and **unlikely to fail due to operating near its limits. :It also takes up a lot of room. ==Materials== * '''Glue''': Hot glue gets used a lot in this project, but not all hot glues are made equal. Most hot glues melt at around 105⁰C which means that, at the temperatures reached while cooking, the glue has softened considerably and everything falls apart. This is both disappointing and the cause of many sarcastic comments from spouses. But all is not lost! You can get high-temperature glue that melts at 165⁰C, which remains stable at sous vide temperatures. High-temperature (and dual-temperature) glue guns are quite common, but the glue is rarer. Make sure you also get glue that only melts at the higher temperature: dual-temperature-compatible glue will still melt at the lower temperature, so won't do. * '''Sealant''': This also gets used a lot in this project. Make sure you use bathroom/kitchen sealant (the kind that smells of vinegar as it cures) rather than the cheaper generic sealant. The generic stuff falls apart when held over hot water for several hours. * '''Switch''': I didn't bother with an additional power switch in the cooker. I just use the wall socket switch to control it. * '''Aluminium project box''': one that's formed from two C-shaped pieces. One half forms the mount for the cooker, the other is sacrificed to make the pump mount. * '''Loop-ended bolts and nuts''': two of these to hold the cooker in place. * '''Zinc sheet''': for sacrificial anodes. Steel in hot water rusts quickly, so I used zinc washers to protect all the submerged nuts and bolts and the heating elements. ==Construction== [[File:Sous-vide-cooker-mk3-inside.jpg|200px|thumb|right|The arrangement of parts inside the box]] # Ensure you have sufficient room on the base of your box for all the components. You'll need the three heaters inline, with the pump outflow pushing water along the centre of the heater coils. The thermocouple should go adjacent to the pump, upstream of the heaters. You'll also need room for the clamp. # Space inside the box is tight, too. Make sure you know where all the components are going. I had to saw off the corner of the base of the relay's heat sink to allow the pump power lead into the box. The 12V power supply has to go on the opposite side of the box from the heaters, which limits places where the power cable can come in from outside. Also, you probably won't be able to mount the controller directly above the heaters, due to lack of vertical space. # Drill the holes in the side of the mounting bracket for the mounting bolts. Use two-part epoxy glue to fix the nuts on the inside of the bracket. # Start by laying out the heaters and pump on the base of the box. Place the heaters as close to the edge as you can while still being able to put glue all round them inside the box. You'll also need to leave room for the mounting bracket. # Make the holes for the heaters and insert them without gluing. Then fix the position of the pump and mounting bracket. You want the pump outflow to be directed through the middle of all three heater coils. Ensure that the pump bracket doesn't overlap the line of where the mounting bracket will be, so that the mounting bracket is the only part that touches the side of the tank. # Make the hole for the thermocouple. # Make holes for the power lead and controller. You may need to trim the lip around the lid of the box where the controller is. # Glue the heaters in place, and screw in the pump bracket, pump, and mounting bracket. I added home-made zinc washers to all the bolts, as sacrificial anodes. # Do all the wiring inside before fixing any components. It'll turn into a bit of a rats nest, but that can't be helped. # Attach the SSR to its heat sink. # Use two-part epoxy glue to secure the SSR heat sink and 12V power supply in place. # The controller stays in place by friction, especially when the lid is secured. ==Wiring== [[File:Tet612-sous-vide-wiring-mk3.png|200px|thumb|right|Wiring for the cooker]] The wiring mostly follows from that in Make. A solid-state relay (SSR) controls power to the heaters. The SSR is controlled by the TET-612 controller with input from the thermocouple. The pump is permanently on while power is supplied to the unit. The connections between the parts are made via screw-down connector blocks. After making all the connections, cover all the terminals with insulation tape to prevent accidental short circuits. Make sure all wires and connectors can take the load: the heaters will draw 1500W at 240VAC, so the unit as a whole could draw over 7A. Not shown on the wiring diagram is an earth connection from the mains to the pump. The heaters have no earth connections. ==Controller parameters== Press set, enter the passcode 0089, and set up the controller parameters as below. {| ! Parameter !! Value !! Comment |- | Inty || J || Input type: Type J thermocouple |- | Outy || 2 || Use the SSR output for control |- | Hy || 0003 || Not used |- | Psb || 0 || Not used |- | Rd || 0 || Controls a heater, not a cooler |- | CorF || 0 || Units: 0 = Celsius |} You might as well also set the alarm thresholds. Enter passcode 0001 and set the alarms as {| ! Parameter !! Value |- | Sv || 62 |- | AH1 || 100 |- | AL1 || 95 |- | AH2 || 85 |- | AL2 || 90 |} ===Controller tuning=== The TET 612 is a [http://en.wikipedia.org/wiki/PID_controller PID controller] and its parameters need to be tuned to control the cooker effectively. The easiest way is for the controller to self-tune by pressing and holding the '>' button until the display flashes. You will need to set the 'SouF' and 'ot' parameters before doing the tuning. The controller will move the water temperature above and below the target three times, so it could take up to an hour to do. To examine or manually adjust the parameters, press set and enter the passcode 0036. Should you be interested, these are the parameters I have. {| !Parameter !! Value |- | P || 0.4 |- | I || 756 |- | d || 189 |- | SouF || 0.3 |- | ot || 2 |- | Filt || 0 |} ==In use== * Don't turn on the cooker unless the heaters are immersed in water above their 'shoulders'. They'll burn out in a very few seconds otherwise. The pump will also burn out quickly if it isn't immersed in water. ==Alternative pumps== The hardest part to sources is a pump that is capable of handling water at 80⁰C+ but still small and cheap enough to be viable the cooker. The P38-B is the best I could find. Alternatives are: * [http://cgi.ebay.com/DC-12V-Electric-Centrifugal-Water-Pump-109-GPH-P-38E-/260688654282?pt=LH_DefaultDomain_0&hash=item3cb2412fca#ht_3194wt_1077 DC 12V Electric Centrifugal Water Pump 109 GPH (P-38E)], an external pump that requires priming * [http://cgi.ebay.com/Mini-Water-Coffee-Pump-24-GPH-P-25A-DC-12V-/250722722777?pt=Small_Kitchen_Appliances_US&hash=item3a603d23d9#ht_2955wt_1141 Mini Water / Coffee Pump 24 GPH (P-25A) DC 12V], a smaller external pump that needs priming * [http://www.lightobject.com/High-temperature100-39C-1LMin-mini-DC-Water-Pump-FDA-approval-Ideal-for-coffee-maker-or-other-food-process-application-P510.aspx High temperature(100 'C) 1L/Min mini DC Water Pump. FDA approval.] (and [http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=160502070921#ht_1161wt_907 on eBay]) a submersible pump that tolerates higher temperature (though only up to 50⁰C submerged). [http://www.tomordway.com/PID_Controller/SousVideController.html Tom Ordway] reports success with this * an aquarium air pump, without an airstone, with the bubbles providing circulation All the pumps require an additional power supply. ==See also== * [http://makeprojects.com/Project/Sous-Vide-Immersion-Cooker/471/1 Original DIY sous vide cooker instructions] * Tom Ordway's [http://www.tomordway.com/PID_Controller/SousVideController.html sous vide cooker]. * Homebrew Talk [http://www.homebrewtalk.com/f51/tet612-pid-133807/ thread on using the TET 612 controller] * [http://auberins.com/index.php?main_page=product_info&cPath=1&products_id=106 Spec page of a TET 612 clone] (with a few more features than the TET 612) * [http://freshmealssolutions.com/downloads/PID-tuning-guide_R2_V006.pdf A guide to PID sous vide parameter tuning] (pdf) [[Category:Sous vide]] 38bce81ef058a3758c77a1815ceb979f80910c9b 6 1620 2387 2011-09-25T10:26:50Z Neil 2 Wiring guide for the sous vide cooker mk 3 wikitext text/x-wiki Wiring guide for the sous vide cooker mk 3 aeace643012ce941f5f3ae079b51f6a4fef92da5 2395 2387 2011-09-25T10:57:23Z Neil 2 wikitext text/x-wiki Wiring guide for the sous vide cooker mk 3. [[Category:Sous vide]] acbde238e670ae368eed79c62c02825d758235b3 6 1621 2389 2011-09-25T10:36:02Z Neil 2 Sous vide cooker mk3 in situ wikitext text/x-wiki Sous vide cooker mk3 in situ 1b847bbec143aa1965ef32b4dab096cf8c388ea4 2394 2389 2011-09-25T10:57:12Z Neil 2 wikitext text/x-wiki Sous vide cooker mk3 in situ. [[Category:Sous vide]] bea28c031fbaf944cb8f03aab297990fee3828f1 6 1622 2396 2011-09-30T20:02:47Z Neil 2 Inside the sous vide cooker mk3 [[Category:Sous vide]] wikitext text/x-wiki Inside the sous vide cooker mk3 [[Category:Sous vide]] 7152446fe2d859b22e894485d3e5b7d851a15907 6 1623 2397 2011-09-30T20:03:24Z Neil 2 Underside of the sous vide cooker mk 3 [[Category:Sous vide]] wikitext text/x-wiki Underside of the sous vide cooker mk 3 [[Category:Sous vide]] 3acff6c71e4f6b45df61fc4e2ded4919ec234bbf 0 1609 2404 2403 2011-10-10T18:52:27Z Neil 2 /* Duck leg confit */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] e4839259629c85473e842b7c9b7dce9d7c3cddd8 2405 2404 2011-10-10T19:00:53Z Neil 2 wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] dc422e3f8219dddfe71d01ff356be59e4abbe244 2406 2405 2011-12-07T15:01:01Z Neil 2 Added Char Sui, butchers wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui (untried)== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (72⁰C). Today was 4 hours at 140F (78⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 78⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] 3114dc3a92ef7eb7394573d7956af7520232eae6 2407 2406 2011-12-09T15:39:21Z Neil 2 /* Pork Char Sui (untried) */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui (untried)== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (72⁰C). Today was 4 hours at 140F (78⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 78⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. (An [http://www.consumedgourmet.com/2011/09/pork-belly-char-siu.html alternative, with a home-made marinade].) ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] 337c707a90c31568416d6038689cfc48341b404d 2408 2407 2011-12-10T09:37:25Z Neil 2 /* Pork Char Sui (untried) */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui (untried)== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (72⁰C). Today was 4 hours at 140F (78⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 78⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. (An [http://www.consumedgourmet.com/2011/09/pork-belly-char-siu.html alternative, with pork belly and a home-made marinade].) ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] 7868e8c4d41bf8e1f5fc8872edf2857c4852012e 2409 2408 2011-12-11T09:32:50Z Neil 2 /* Pork Char Sui (untried) */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui (untried)== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (55⁰C). Today was 4 hours at 140F (60⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 60⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. (An [http://www.consumedgourmet.com/2011/09/pork-belly-char-siu.html alternative, with pork belly and a home-made marinade].) ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] d7bc720d4f1a47d2ea49b1f06cc03cf4f66b0ebe 2410 2409 2011-12-11T15:52:54Z Neil 2 /* Pork Char Sui (untried) */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (55⁰C). Today was 4 hours at 140F (60⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 60⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. (An [http://www.consumedgourmet.com/2011/09/pork-belly-char-siu.html alternative, with pork belly and a home-made marinade].) :'''Verdict''': I tried it with the home-made marinade over pork tenderloin at 60⁰C for four hours. It was great. Lots of flavour from the marinade and the pork was lovely and tender. I had to use a frying pan to char the meat afterwards, and that wasn't too successful. But it didn't detract from the flavour of the dish. ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] 2ec2afc91c8e8aba5ab9d8e6012d9689d08de404 2411 2410 2011-12-23T18:07:05Z Neil 2 /* Pork Char Sui */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (55⁰C). Today was 4 hours at 140F (60⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 60⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. An [http://www.consumedgourmet.com/2011/09/pork-belly-char-siu.html alternative, with pork belly and a home-made marinade]. This is sufficient for two tenterloins * Garlic, minced: 5 grams * Sugar: 15 grams * Five spice: 1 gram * Hoisin: 25 grams * Honey: 20 grams * Shaoxing wine: 15 grams * Usukuchi (light) soy: 15 grams * Sesame oil: 15 grams Combine the ingredients, spread over the tenderloins, and marinade for 12 hours at least. Then cook for 4 hours at 60⁰C, remove and pat dry, and baste with another batch of marinade before frying or grilling. :'''Verdict''': I tried it with the home-made marinade over pork tenderloin at 60⁰C for four hours. It was great. Lots of flavour from the marinade and the pork was lovely and tender. I had to use a frying pan to char the meat afterwards, and that wasn't too successful. But it didn't detract from the flavour of the dish. ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet] * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] d2a6b5413307bbd9f29bfa1d06f9d2005cbe97e7 2413 2411 2012-02-08T11:02:06Z Neil 2 /* See also */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (55⁰C). Today was 4 hours at 140F (60⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 60⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. An [http://www.consumedgourmet.com/2011/09/pork-belly-char-siu.html alternative, with pork belly and a home-made marinade]. This is sufficient for two tenterloins * Garlic, minced: 5 grams * Sugar: 15 grams * Five spice: 1 gram * Hoisin: 25 grams * Honey: 20 grams * Shaoxing wine: 15 grams * Usukuchi (light) soy: 15 grams * Sesame oil: 15 grams Combine the ingredients, spread over the tenderloins, and marinade for 12 hours at least. Then cook for 4 hours at 60⁰C, remove and pat dry, and baste with another batch of marinade before frying or grilling. :'''Verdict''': I tried it with the home-made marinade over pork tenderloin at 60⁰C for four hours. It was great. Lots of flavour from the marinade and the pork was lovely and tender. I had to use a frying pan to char the meat afterwards, and that wasn't too successful. But it didn't detract from the flavour of the dish. ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet (thread 1)] and [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/ thread 2]. * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] b3a860eae56e2c7726d3bf0fa941cf27b1911ba6 2422 2413 2012-04-28T10:09:41Z Neil 2 wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)] (untried)== (US short ribs are UK thin ribs) ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (55⁰C). Today was 4 hours at 140F (60⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 60⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. An [http://www.consumedgourmet.com/2011/09/pork-belly-char-siu.html alternative, with pork belly and a home-made marinade]. This is sufficient for two tenterloins * Garlic, minced: 5 grams * Sugar: 15 grams * Five spice: 1 gram * Hoisin: 25 grams * Honey: 20 grams * Shaoxing wine: 15 grams * Usukuchi (light) soy: 15 grams * Sesame oil: 15 grams Combine the ingredients, spread over the tenderloins, and marinade for 12 hours at least. Then cook for 4 hours at 60⁰C, remove and pat dry, and baste with another batch of marinade before frying or grilling. :'''Verdict''': I tried it with the home-made marinade over pork tenderloin at 60⁰C for four hours. It was great. Lots of flavour from the marinade and the pork was lovely and tender. I had to use a frying pan to char the meat afterwards, and that wasn't too successful. But it didn't detract from the flavour of the dish. ==Roast (leg of) pork== * Remove the skin for crackling, leaving as much fat as possible on the skin. * Bag up the meat with a little bit of sage. * Cook at 70⁰C for 24 hours. * Dry the skin, score, and rub in about a dessertspoon of salt. Wrap the skin in a teatowel and keep in the fridge for 23 hours. * 45 mins before serving, preheat the oven to 220⁰C. * Place the skin on a baking sheet in the oven. After 10 mins, turn the heat down to 170⁰C. * After cooking for 24 hours, remove the pork, dry off, and sear in a very hot pan. * Reserve the pork juices for gravy. * Slice and serve with the crackling. :'''Verdict:''' The crackling was fantastic. The pork was nicely done and quite tender, though it was a little overcooked. As leg of pork isn't a tough cut, I'll try it at 60⁰C for 24 hours next time. ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet (thread 1)] and [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/ thread 2]. * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] 1f5f707ea823e6c30ab156b0661bb91145f4837a 2427 2422 2012-06-15T19:09:33Z Neil 2 /* 48 hour Sous Vide Short Ribs (Momofuku) (untried) */ wikitext text/x-wiki Some recipes, and links to other recipes. ==Food safety notes== You need to make sure that the bags are fully immersed in the water throughout the cooking time. If you're using manual-seal bags, make sure that you remove as much air as possible to get a good thermal contact between the food and the water. I use small metal cooling racks and metal spoons placed on top of the food bags to keep them immersed. Note that some water in the food may evaporate during cooking, causing the bag to float more after a while, so make sure the bags are well weighed down. If you're at all worried about the state of the food, throw it out. It's not worth eating stuff that's been happily breeding food pathogens (and accumulating their toxins) in perfect growing conditions for several hours. ==Beef steaks== Marinade in red wine and crushed garlic before. Dry the steaks and pat dry before placing them in a vacuum sealer bag. Cook at 55⁰C-58⁰C for 60 minutes. '''Verdict:''' Don't overdo the marinade, either in flavour or length of time. The flavours don't diminish with cooking, unlike traditional frying. ==Chicken breasts== 60⁰C-63⁰C, 60 minutes. '''Verdict:''' Excellent chicken, moist with a stronger chicken flavour than traditionally-cooked chicken. Some people liken the consistency to processed chicken. ==Duck leg confit== * Does two duck legs. * Crush a couple of bay leaves into a bowl. Add a couple of teaspoons of dried thyme and a tablespoon of salt. Mix and rub into the legs, particularly into the meat. :(Alternatively, use about ¼ teaspoon of five spice powder per leg instead of the thyme and bay. You still need the salt.) * Store in the fridge for 24 hours. * Rinse ''thoroughly'' under cold water. You may need to remove the membrane from the meat if the herbs stick to it. * Place each leg into a separate bag. There's no need to add additional duck fat, but adding a tablespoon per leg won't hurt. (I don't bother.) * Cook at 74⁰C for 20 hours. * After cooking, remove the skin and shallow fry for a few minutes to crisp it up. * Serve together with the shredded meat. * Reserve the liquid from the bags: it's more duck fat and gelatine. '''Verdict:''' Sublime. Just sublime. ==Glazed carrots (don't)== '''Warning!''' Do not use this recipe in a home-made SV setup that has plastic parts in the water bath (such as an aquarium pump as an agitator, or a perspex support). The hot water will melt the plastic parts, so that your pump will no longer work and the support will melt. You have been warned! * Wash and cut 500g carrots into 2cm chunks (baby carrots can be left whole). * Seal with 2 tbsp butter and 1 tbsp sugar, and perhaps a bit of pepper. * Cook for 1 hour at 84⁰C. * After cooking, carrots can be kept in the bag in the fridge for up to a week. * Empty contents of bag into a hot frying pan. Cook over a high heat until all the water has evaporated and the carrots are glazed. (Recipe from [http://www.seriouseats.com/recipes/2010/06/sous-vide-glazed-carrots-recipe.html Serious Eats].) '''Verdict:''' Don't do this. The carrots were nice, but not sufficiently nice to justify the damage to the cooker. ==Retrograde starch mashed potatoes== * Slice potatoes and cook at 66⁰C for 30 minutes * Cool rapidly under running cold water * Cook normally in boiling, not simmering, water * Mash, and it won't end up gloopy '''Verdict:''' First attempt at 70⁰C was OK, but the dryer texture of the mash wasn't what people were used to, so they preferred the traditional boiled potatoes. I couldn't tell a great deal of difference, it must be said. ==Aspargus== * Place in bag with salt, pepper, and a little butter * Cook at 65⁰C for 8-15 minutes (opinions vary on the correct time) '''Verdict:''' The asparagus was cooked, but still very crunchy. We all preferred the more traditional, steamed asparagus. ==[http://online.wsj.com/article/SB10001424052748704107204574473292205193888.html Marinated Hanger Steak Ssäm] (untried)== ==[http://www.tinyurbankitchen.com/2010/09/48-hour-sous-vide-short-ribs-momofuku.html 48 hour Sous Vide Short Ribs (Momofuku)]== (US short ribs are UK thin ribs) '''Verdict:''' OK, but nothing special. ==[http://svkitchen.com/?p=2582 Lamb shanks with port] (untried)== ==Leg of Mutton== No aromatics beyond a little salt and pepper, maybe a smattering of garlic. 58⁰C for 24 hours. Flash-fry in a pan afterwards to brown the outside. The version from [http://www.kamikazecookery.com/blogs/184 Kamikaze Cookery] recommends cooking at 70⁰C, but that's too hot. It just overcooks the muscle tissue, leaving it dry. '''Verdict:''' Just fabulous. The mutton was wonderfully moist and tender, with loads of flavour. And that flavour was spectacular! ==Pork ribs (untried)== * Marinade in 7% salt solution for 24 hours. * Dry and add dry rub. Seal. * Cook at 63⁰C for 24 hours. * Dry and fry, preferably deep-fry for a few moments to crisp. Using [http://www.douglasbaldwin.com/sous-vide.html#Pork Baldwin's recipe]. ==Pork Char Sui== Details from an [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/page__st__1080__p__1851933#entry1851933 eGullet post]. Previously I had cooked it for 4 hours at 130F (55⁰C). Today was 4 hours at 140F (60⁰C). It was, of course, less pink, but still tender and juicy and we all liked the texture much better. So this is the way I will be doing it from now on. To recap, I start with pork tenderloin. Rub it liberally with a packaged char siu powder (normally intended to be mixed with water to form a marinade) then vacuum seal it and refrigerate for about 12 hours. Then into the bath for 4-5 hours at 60⁰C. Remove from bath and pat dry, then baste on all sides with honey. Char it on a very hot BBQ for a few minutes on each side (or you can do it under a broiler). Remove and rebaste with honey and allow to rest a while, then slice. An [http://www.consumedgourmet.com/2011/09/pork-belly-char-siu.html alternative, with pork belly and a home-made marinade]. This is sufficient for two tenterloins * Garlic, minced: 5 grams * Sugar: 15 grams * Five spice: 1 gram * Hoisin: 25 grams * Honey: 20 grams * Shaoxing wine: 15 grams * Usukuchi (light) soy: 15 grams * Sesame oil: 15 grams Combine the ingredients, spread over the tenderloins, and marinade for 12 hours at least. Then cook for 4 hours at 60⁰C, remove and pat dry, and baste with another batch of marinade before frying or grilling. :'''Verdict''': I tried it with the home-made marinade over pork tenderloin at 60⁰C for four hours. It was great. Lots of flavour from the marinade and the pork was lovely and tender. I had to use a frying pan to char the meat afterwards, and that wasn't too successful. But it didn't detract from the flavour of the dish. ==Roast (leg of) pork== * Remove the skin for crackling, leaving as much fat as possible on the skin. * Bag up the meat with a little bit of sage. * Cook at 70⁰C for 24 hours. * Dry the skin, score, and rub in about a dessertspoon of salt. Wrap the skin in a teatowel and keep in the fridge for 23 hours. * 45 mins before serving, preheat the oven to 220⁰C. * Place the skin on a baking sheet in the oven. After 10 mins, turn the heat down to 170⁰C. * After cooking for 24 hours, remove the pork, dry off, and sear in a very hot pan. * Reserve the pork juices for gravy. * Slice and serve with the crackling. :'''Verdict:''' The crackling was fantastic. The pork was nicely done and quite tender, though it was a little overcooked. As leg of pork isn't a tough cut, I'll try it at 60⁰C for 24 hours next time. ==Ingredient sources== [http://jarvisbutchers.wordpress.com/contact-us/ Jarvis Butchers] are a good source of the various odd cuts of meat, as is the butcher's counter at Morrisons. ==See also== * Jeff Potter's [http://www.cookingforgeeks.com/ Cooking for Geeks] website. * Douglas Baldwin's [http://www.douglasbaldwin.com/sous-vide.html Guide to sous vide cooking], including time/temperature tables. * A [http://www.sousvidecooking.org/ sous vide cooking blog]. * [http://www.seriouseats.com/2010/03/how-to-sous-vide-steak.html Sous vide steak cooking] * [http://forums.egullet.org/index.php?/topic/136274-sous-vide-index/ Sous vide index on eGullet (thread 1)] and [http://forums.egullet.org/index.php?/topic/136275-sous-vide-recipes-techniques-equipment-2011/ thread 2]. * [http://www.kamikazecookery.com/blogs/134 12 tips for sous vide goodness] * [http://svkitchen.com/ SV Kitchen], with recipes and tips * [http://www.google.co.uk/search?q=sous+vide+recipe+site%3Anomnompaleo.com Sous vide recipes] from [http://nomnompaleo.com/ Nom Nom Paleo] ([http://nomnompaleo.com/recipeindex all recipes]) [[Category:Sous vide]] c56c4f768e25ddfd035a778dc700425089537b78 0 1541 2412 2366 2012-01-09T15:07:21Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/resolv.conf</code> to include domain domain.tld search domain.tld :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 * Change <tt>/etc/resolv.conf</tt> to nameserver 192.168.1.1 (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] 5324bc77f744f79159c4b86e13f23155fd25d667 0 1549 2414 1819 2012-02-10T14:12:48Z Neil 2 wikitext text/x-wiki ''(This was written by Michael Harvey to support his [[:Category:Name generation|name generation]] program.)'' Made-up names, especially those found in literature and roleplaying games, often sound silly, out of place, or not-quite right. To be honest, it can be difficult thinking up creative names that sound right. Names is a program which can generate hundreds of names based on the input you feed it. Because it combines "elements" from data files to form names, it is possible to generate very high quality output. Further, it is possible to create data files which provide a "feel" for a particular language or culture -- real or imaginary. This enables the writer or game designer to generate lists of names for each fantasy culture, which all have a certain sound and seem to belong together. This document has various suggestions on how to create element files, as well as how to generate good quality names. == Names elements == There are several types of name elements: ;Prefix (PRE) :Parts that begin a word. Example: "c", "co", and "con" could all be prefixes for "Conan" ;Middle (MID) :Parts that go in the middle of a word, ''between'' a prefix and suffix. Example: "cam" is a middle for "Alcamtar", where "al" is the prefix and "tar" is the suffix ;Suffix (SUF) :Parts that end a word. In the name Aragorn, suffixes could be "agorn", "gorn", "orn", or even "n" ;Noun (NOUN) :Nouns: "road", "fortress", "tree", "crossing" ;Adjective (ADJ) :Adjectives: "gray", "slow", "stubborn" ;Noun/adjective (NADJ) :These are words which could be either nouns or adjectives. Examples: dark ("darkwood", "outer dark"), silver ("silverlode", "moria-silver"), wood ("woodtown", "goblin-wood") :Many nouns and adjectives actually fall into this category. This gives the program more flexibility in using them, providing much more variety in names. Elements should not be duplicated in two categories. For example, "orc" should not be under both noun and adjective, or you might get something like "Orcorc". Instead, make it a NADJ and it won't get used twice. Elements that you want to emphasize may be duplicated within the same category. For instance, if you list "forest" three times in a row, it is three times more likely to be selected. Elements are stored in plain text .ELE files. Thus, you might have one file of goblin sounding names and another for elvish words, etc. == Element files == These should have the extension <code>.ELE</code>. The element file contains several types of elements, denoted by simple abbreviations: {| |<code>PRE</code>||prefixes |- |<code>MID</code>||middles |- |<code>SUF</code>||suffixes |- |<code>NOUN</code>||nouns |- |<code>ADJ</code>||adjectives |- |<code>NADJ</code>||noun/adjectives |- |<code>END</code>||this signifies the end of the file |} The pound sign (#) is used as an escape character to denote the start of a new section (much like preprocessor directives in C). Thus, a typical file would look like this: #PRE con gal tar #SUF an arok #END Example output: Conan Galan Taran Galarok Conarok Tararok This defines three prefixes (con,gal,tar) and two suffixes (an,arok). Notice the #END used to end the file. There should not be any blank lines in the file, unless you want to define empty parts. Parts may include any character, including spaces. For example, #PRE con gal tar #SUF an 'Arok -Rog e arba zud #END Might produce the following output: Con An Gal'Arok Galzud Tar-Rog Cone Arba Elements in an element file should not be capitalized. The program will automatically capitalize the names after each space. If you do put any capitals in, they will not be converted to lowercase, so this could be used to artificially capitalize words (usually after special punctuation, as above). The exception to this general "rule" is when you want to capitalize an element after a punctuation mark; in the example above, 'Arok would not have been automatically capitalized because there is no whitespace. I find it helpful to keep all the elements in each section alphabetized so that I can check for duplicates. == Use of element files == The prefix/suffix form is generally the most useful. Most of the example files are in this form. Note that you are not restricted to fantasy names. By using english words you can get things like "Darkwood", "Brightblade", "Millennium Falcon", "Nottingham" and so forth. See the sample files for ideas. Middles can add a lot of variety to your names, but they also make the names themselves longer. Noun/NADJ/ADJ elements are mostly useful for idea-generators, or special places like "House of the Skulls" or "The Books of the Priest." The program combines these in a variety of ways, and also pluralizes them sometimes. All you need to do is supply the list of words. Such lists may represent themes, such as dark fantasy, science fiction, nautical things, wizardly things, and so forth. Again, see the sample ELE files. It is easy to make your own ELE files. There are several methods I have found personally useful. One is to just sit down and starting making up the elements out of your head. Just come up with all the combinations you can think of and put them in. Another is to compile a list of names from your favorite book(s), movie(s), etc. You can then easily split these names into prefix/suffix parts. Example: {| ! Original name !! PRE !! SUF |- | Alcamtar || alcam || tar |- | Celowin || cel || owin |- | Conan || con || an |- | Gimli || gim || li |- | Talward || tal || ward |} One helpful strategy in making your names sound better is to end the prefixes and begin the suffixes consistently. Either end the prefixes with a vowel and begin the suffixes with a consonant, or end the prefixes with a consonant and begin the suffixes with a vowel. I generally prefer the latter. Example: {| ! !! Method #1 !! Method #2 |- | Alcamtar || alca / mtar || alcam / tar |- | Celowin || celo / win || cel / owin |- | Conan || co / nan || con / an |- | Gimli || gi / mli || gim / li |- | Talward || talwa / rd || tal / ward |} This technique will keep your names from sounding too random, and will make them more pronounceable. Note that soft sounds such as r, l, t, s, w, y might be including with the vowels, as I have done above. Use your imagination and common sense, and put a few together yourself to see how they sound. Another helpful technique is to make all the words of a given language use the same set of sounds, or same spelling. This might require a little research. Some good references are the appendices to the Lord of the Rings, or an introductory book on phonetics/language from the library. Some simple examples: {| | Elvish spelling: || 'C' || 'G' || 'DH' || 'T' || 'S' || 'W' || 'F' |- | Orcish spelling: || 'K' || 'GH' || 'J' || 'D' || 'Z' || 'V' || 'V' |} Here is a chart. I'm not going to take the time to try an explain it, but study it and compare the sounds to one another. You will notice patterns. It probably isn't too accurate, but is still useful. (Yes, I got the idea from Tolkein.) {| ! !!colspan=2| STOPS !!colspan=2| SPIRANTS !!colspan=2| OTHER !!colspan=2| NASAL |- ! !! Soft !! Voiced !! Soft !! Voiced !! Soft !! Voiced !! Soft !! Voiced |- ! Labial |p || b || || bh || wh || w || mh || m |- ! Labiodental | || || f || v || || y || || |- ! Dental | || || th || th || || || || |- ! Palatal? | t || d || ch || j || lh || l || nh || n |- ! Sibilant | s || z || sh || zh || rh || r || || |- ! Alveolar? | c,q,k || g || ch || gh || h || h || || ng |} You can use this to create "sounds" for languages. For instance, celtic has a lot of unvoiced (soft) sounds, making it sound smooth and soft and flowing. German, on the other hand, has a lot of voiced stops, making it louder and harsher sounding. By selecting which sounds are most common, you can give each fantasy language a certain "feel." One handy technique is to get an atlas of foreign countries and look at the names on the map. You can get a pretty good idea of which sounds are common, and how they should go together this way. This is the way I did my "french" ELE file, and also the "aztec" and "african" ones. == Conclusion == Hope you find this useful. I can be contacted for comments/questions on the Internet as "mike@cs.pdx.edu". [[Category:Name generation]] b8a48c265c34cd3601da55546cd3f8ee5930bd85 0 1526 2415 2265 2012-02-15T15:23:46Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>apt-get autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/apt-get-autoclean</code>: #! /bin/sh # apt-get-autoclean Cron script to clear apt cache of unneeded files. # apt-get autoclean == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * texlive-latex-base * texlive-latex-extra * texlive-science * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Mode switch key, and the 'right-click menu' key as the Compose key. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. For instance AltGr+5 will generate ½. <code>~/.Xmodmap</code> has the details. The Compose key allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 48ecf2a1b3c954aa573a5f0e1a39ad1ee9f470e2 2423 2415 2012-05-02T09:52:02Z Neil 2 /* Enable the Compose key and set AltGr key as Mode_swich */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>apt-get autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/apt-get-autoclean</code>: #! /bin/sh # apt-get-autoclean Cron script to clear apt cache of unneeded files. # apt-get autoclean == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * texlive-latex-base * texlive-latex-extra * texlive-science * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 30355f02478dccddc309359cb25e08097b15df55 0 1 2416 2382 2012-03-14T15:45:11Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == {| align="right" {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/116086712514085162080/about Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github |} Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. b4c7ee5296e2183f4d856e5d3266f2e3d254590b 4 1624 2417 2012-04-05T16:48:55Z Neil 2 Created page with 'Empty page.' wikitext text/x-wiki Empty page. e8091188011bf06a8a868bdfb483364639b3e7ce 4 1625 2418 2012-04-05T16:49:13Z Neil 2 Created page with 'Empty page.' wikitext text/x-wiki Empty page. e8091188011bf06a8a868bdfb483364639b3e7ce 4 1626 2419 2012-04-05T16:49:20Z Neil 2 Created page with 'Empty page.' wikitext text/x-wiki Empty page. e8091188011bf06a8a868bdfb483364639b3e7ce 4 1627 2420 2012-04-05T17:02:56Z Neil 2 Created page with 'Empty page.' wikitext text/x-wiki Empty page. e8091188011bf06a8a868bdfb483364639b3e7ce 4 1628 2421 2012-04-05T17:03:05Z Neil 2 Created page with 'Empty page.' wikitext text/x-wiki Empty page. e8091188011bf06a8a868bdfb483364639b3e7ce 0 1544 2424 2367 2012-06-15T10:02:46Z Neil 2 /* Chrooting the nameservers */ Added OpenSSL fix for Ubuntu 12.04 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Disable rndc by adding this line to the top of <code>/etc/bind/named.conf</code> controls { }; :I could never get it working, and always use the <code>/etc/init.d/bind9</code> init script to control bind. * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 192.168.2.251 nameserver 192.168.2.252 :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == For increased security, it's now time to get the nameservers running in a chroot jail so that they can't see the rest of the filesystem. Second, create the slave nameserver on Desktop. BIND on Fedora Core 3 is automatically run in a chroot jail. The [https://help.ubuntu.com/community/BIND9ServerHowto#head-b9cfe2895d instructions on the Ubuntu wiki] didn't quite work for me. * Create the chroot environment: root@server:~# mkdir -p /chroot/bind/dev root@server:~# mkdir -p /chroot/bind/etc/bind root@server:~# mkdir -p /chroot/bind/var/run/named root@server:~# mkdir -p /chroot/bind/var/cache/bind * Copy the BIND config files into the chroot jail: root@server:~# cp /etc/bind/* /chroot/bind/etc/bind * Create the devices BIND requires: root@server:~# mknod /chroot/bind/dev/null c 1 3 root@server:~# mknod /chroot/bind/dev/random c 1 8 * Set the file ownerships and permissions: root@server:~# chown -R bind:bind /chroot/bind/etc root@server:~# chown -R bind:bind /chroot/bind/var/run root@server:~# chown -R bind:bind /chroot/bind/var/cache * Modify the file <code>/etc/default/bind9</code> so that BIND now runs in a chroot jail: OPTIONS="-u bind -t /chroot/bind" # Set RESOLVCONF=no to not run resolvconf RESOLVCONF=yes * '''For Ubuntu 8.04:''' Modify <code>/etc/default/syslogd</code> to log messages from the chrooted BIND: SYSLOGD="-u syslog -a /chroot/bind/dev/log" * '''For Ubuntu 10.04:''' Create <code>/etc/rsyslog.d/bind-chroot.conf</code> to log messages from the chrooted BIND: $AddUnixListenSocket /chroot/bind/dev/log * '''For Ubuntu 12.04:''' Copy the required OpenSSL library into the chroot jail: root@server:~# mkdir -p /chroot/bind/usr/lib/i386-linux-gnu/openssl-1.0.0/engines root@server:~# cp /usr/lib/i386-linux-gnu/openssl-1.0.0/engines/libgost.so /chroot/bind/usr/lib/i386-linux-gnu/openssl-1.0.0/engines/ * Make AppArmor allow BIND access to the chroot jail. Modify <code>/etc/apparmor.d/usr.sbin.named</code> to include the lines: # Additional permissions for a chrooted bind /chroot/bind/etc/bind/** r, /chroot/bind/var/lib/bind/** rw, /chroot/bind/var/lib/bind/ rw, /chroot/bind/var/cache/bind/** rw, /chroot/bind/var/cache/bind/ rw, /chroot/bind/var/run/named/named.pid w, /chroot/bind/var/run/named/session.key w, # support for resolvconf /chroot/bind/var/run/bind/named.options r, # Allow access to copies of OpenSSL libraries (Ubuntu 12.04) /chroot/bind/usr/lib/i386-linux-gnu/openssl-1.0.0/engines/** rm, /chroot/bind/usr/lib/i386-linux-gnu/openssl-1.0.0/engines/ rm, * Restart AppArmor, system logger, and BIND: root@server:~# /etc/init.d/apparmor restart ** '''For Ubuntu 8.04:''' root@server:~# /etc/init.d/sysklogd restart ** '''For Ubuntu 10.04:''' root@server:~# service rsyslog restart * For both: root@server:~# /etc/init.d/bind9 restart * Check <code>/var/log/syslog</code> for messages of BIND restarting, loading zone files, sending and receiving notifications, and so on. Check that sample queries still work. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 33ac067037ee2b458f866139bf34d38348f595a4 0 1534 2425 2262 2012-06-15T10:43:31Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Restart Apache root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></cocde> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <code>update.php</code> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <code>AdminSettings.php</code> file. Also, with a wiki farm, I'm not sure how to it will know which database to update. Instead, do the following: * Take a backup of the wiki database * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Create the <code>config</code> directory and copy the contents over from <code>/var/lib/mediawiki/config/</code> root@server:~# mkdir /var/www/site.domain.tld/mediawiki/config root@server:~# cp /var/lib/mediawiki/config/* /var/www/site.domain.tld/mediawiki/config/ root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/mediawiki/config * With a browser, visit <code><nowiki>http://site.domain.tld/mediawiki/index.php</nowiki></code> and follow the setup instructions. Remember to include the password for the MySQL root user, so the update script can create or modify any tables it needs to. * Remove any <code>LocalSettings.php</code> and the copy of the <code>config</code> directory * Rename <code>LocalSettings.php.old</code> to <code>LocalSettings.php</code> == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] dad9d0b634bc17f0d95bec86bceae4d170894b9a 2426 2425 2012-06-15T12:07:32Z Neil 2 /* Creating the farm */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Upgrading from previous versions == The [http://www.mediawiki.org/wiki/Manual:Upgrading default way to do this] is to run the <code>update.php</code> script. Trouble is, that doesn't work because Ubuntu seem to have misplaced the <code>AdminSettings.php</code> file. Also, with a wiki farm, I'm not sure how to it will know which database to update. Instead, do the following: * Take a backup of the wiki database * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Create the <code>config</code> directory and copy the contents over from <code>/var/lib/mediawiki/config/</code> root@server:~# mkdir /var/www/site.domain.tld/mediawiki/config root@server:~# cp /var/lib/mediawiki/config/* /var/www/site.domain.tld/mediawiki/config/ root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/mediawiki/config * With a browser, visit <code><nowiki>http://site.domain.tld/mediawiki/index.php</nowiki></code> and follow the setup instructions. Remember to include the password for the MySQL root user, so the update script can create or modify any tables it needs to. * Remove any <code>LocalSettings.php</code> and the copy of the <code>config</code> directory * Rename <code>LocalSettings.php.old</code> to <code>LocalSettings.php</code> == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 929f593b6f6525ffa31ae7ece06dad9f29e81717 0 1516 2428 2206 2012-09-27T19:10:47Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Router setup]] |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/business/server/overview Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <code>build-essentials</code> (for basic software development) and <code>openssh-server</code> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. ==Partitioning the root filesystem (with LVM)== For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a few GB each for <code>/var/log</code> and <code>/tmp</code>, and everything else just goes into one large partition. However, the installer doesn't let you create those partitions very easily, so I ended up doing it manually. == Install proprietary graphics card driver == I have a NVidia GeForce G210 video card. The proprietary video card driver uses the card's GPU for better graphics; the open source one makes KDE hang all too often. This means you need to install the proprietary driver as soon as the OS is installed on the desktop machine. To check that the card is supported, do: root@desktop:~# lspci | grep -i nvidia 02:00.0 VGA compatible controller: nVidia Corporation GT218 [GeForce G210] (rev a2) note the code at the start of the line, and use that to find the PCI ID of the card: root@desktop:~# lspci -n | grep 02:00 02:00.0 0300: 10de:0a60 (rev a2) and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:0a60</code>; the GeForce 6600 on the old machine was <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu -> System -> Hardware Drivers Manager (in Kubuntu 10.04, it's K -> Applications -> System -> Hardware Drivers) and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. [[Category:Server setup]] 13476903d4549fdafe419b44b8c5e863850f1e1a 2429 2428 2012-09-27T20:10:02Z Neil 2 /* Partitioning the root filesystem (with LVM) */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Router setup]] |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/business/server/overview Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <code>build-essentials</code> (for basic software development) and <code>openssh-server</code> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. ==Partitioning the root filesystem (with LVM)== For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a few GB each for <code>/var/log</code> and <code>/tmp</code>, and everything else just goes into one large partition. However, for reasons that aren't that clear any more, I couldn't get the installer to create those partitions, so I had to do it manually. This was complicated by the fact that I wanted to resize the root filesystem, which you can't do when it's mounted. The way around it is to boot the system, after installation, with the Ubuntu Live CD. (This requires that the machine has a keyboard, mouse, and monitor attached. Something to bear in mind if you're installing Ubuntu on a headless server.) Once you've started Ubuntu, use the "Try Ubuntu" option. Open a terminal and install the LVM tools user@subuntu:~$ sudo -i root@ubuntu:~# apt-get install lvm2 Then, create a mount point and mount the root file system: root@ubuntu:~# mkdir /media/rootfs root@ubuntu:~# mount /dev/server/root == Install proprietary graphics card driver == I have a NVidia GeForce G210 video card. The proprietary video card driver uses the card's GPU for better graphics; the open source one makes KDE hang all too often. This means you need to install the proprietary driver as soon as the OS is installed on the desktop machine. To check that the card is supported, do: root@desktop:~# lspci | grep -i nvidia 02:00.0 VGA compatible controller: nVidia Corporation GT218 [GeForce G210] (rev a2) note the code at the start of the line, and use that to find the PCI ID of the card: root@desktop:~# lspci -n | grep 02:00 02:00.0 0300: 10de:0a60 (rev a2) and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:0a60</code>; the GeForce 6600 on the old machine was <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu -> System -> Hardware Drivers Manager (in Kubuntu 10.04, it's K -> Applications -> System -> Hardware Drivers) and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. [[Category:Server setup]] 797648d7bf7cbfffb8d3e98ae1a008b7dfb40ff0 2431 2429 2012-10-03T15:12:58Z Neil 2 Updated LVM section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Router setup]] |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/business/server/overview Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <code>build-essentials</code> (for basic software development) and <code>openssh-server</code> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. ==Partitioning the root filesystem (with LVM)== For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a few GB each for <code>/var/log</code> and <code>/tmp</code>, and everything else just goes into one large partition. However, for reasons that aren't that clear any more, I couldn't get the installer to create those partitions, so I had to do it manually. This was complicated by the fact that I wanted to resize the root filesystem, which you can't do when it's mounted. The way around it is to boot the system, after installation, with the Ubuntu Live CD. (This requires that the machine has a keyboard, mouse, and monitor attached. Something to bear in mind if you're installing Ubuntu on a headless server.) Once you've started Ubuntu, use the "Try Ubuntu" option. * Open a terminal and install the LVM tools: user@subuntu:~$ sudo -i root@ubuntu:~# apt-get install lvm2 * Find the name of the root filesystem LVM: root@ubuntu:~# lvs root@ubuntu:~# vgchange --available y server * Resize the logical volume containing the root filesystem, including resizing the filesystem there: lvresize --resizefs --size -50G /dev/server/root * Now you have some space, you can resize the swap logical volume root@ubuntu:~# swapoff -v /dev/server/swap root@ubuntu:~# lvresize /dev/server/swap -L +2G root@ubuntu:~# mkswap /dev/server/swap root@ubuntu:~# swapon -va :(You won't need the <code>swapon</code> and <code>swapoff</code> commands if you're doing it from the LiveCD.) * Create two new logical volumes, for <code>/var/log</code> and <code>/tmp</code>, and add filesystems to them: root@ubuntu:~# lvcreate -L 20G -n tmp server root@ubuntu:~# lvcreate -L 25G -n varlog server root@ubuntu:~# mkfs.ext4 /dev/server/tmp root@ubuntu:~# mkfs.ext4 /dev/server/varlog * Create mount points and mount the <code>root</code>, <code>varlog</code> partition, and <code>tmp</code> partitions: root@ubuntu:~# mkdir /media/rootfs root@ubuntu:~# mount /dev/server/root root@ubuntu:~# mkdir /media/varlog root@ubuntu:~# mount /dev/server/varlog root@ubuntu:~# mkdir /media/tmp root@ubuntu:~# mount /dev/server/tmp * Copy the existing contents of <code>/var/log</code> to the new logical volume: root@ubuntu:~# cp -pr basefs/var/log/* varlog/ * Modify <code>/etc/fstab</code> to use the new volumes. Add these lines immediately after where the root filesystem is mounted: /dev/mapper/server-tmp /tmp ext4 defaults 0 1 /dev/mapper/server-varlog /var/log ext4 defaults 0 1 * Remove the LiveCD and reboot. * After booting, change the permissions on /tmp: chmod a+rwx /tmp chmod o+t /tmp :(You may have problems with [[MySQL configuration|MySQL]], so that my need reinstalling.) (See more on [http://linuxconfig.org/Linux_lvm_-_Logical_Volume_Manager#5-create-logical-volumes LVM management] and [http://www.centos.org/docs/4/4.5/System_Administration_Guide/Adding_Swap_Space-Extending_Swap_on_an_LVM2_Logical_Volume.html Extending swap on LVM].) == Install proprietary graphics card driver == I have a NVidia GeForce G210 video card. The proprietary video card driver uses the card's GPU for better graphics; the open source one makes KDE hang all too often. This means you need to install the proprietary driver as soon as the OS is installed on the desktop machine. To check that the card is supported, do: root@desktop:~# lspci | grep -i nvidia 02:00.0 VGA compatible controller: nVidia Corporation GT218 [GeForce G210] (rev a2) note the code at the start of the line, and use that to find the PCI ID of the card: root@desktop:~# lspci -n | grep 02:00 02:00.0 0300: 10de:0a60 (rev a2) and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:0a60</code>; the GeForce 6600 on the old machine was <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu -> System -> Hardware Drivers Manager (in Kubuntu 10.04, it's K -> Applications -> System -> Hardware Drivers) and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. [[Category:Server setup]] 89e1c3a37b7f9caa959fc15d18f52ea413d83db5 0 1540 2430 2222 2012-09-28T08:08:49Z Neil 2 /* Anacron timing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 0f121b86f4bf728cf49b5236447a9b74837a5b70 0 1515 2432 2261 2012-10-03T15:14:05Z Neil 2 Swapped order of Dovecot and Postfix setups wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 10.04 LTS (adapted from 8.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Subversion setup|Version control system server (Subversion)]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 8e6098fb460e8a4f26171f19a225b5cdfa56255f 0 1531 2433 2267 2012-10-03T15:24:17Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update <tt>/etc/postfix/main.cf</tt> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <tt>/etc/postfix/vhosts</tt> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <tt>/etc/postfix/vmaps</tt> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <tt>/etc/postfix/vmaps</tt> is changed) * Create aliases for the users in <tt>/etc/postfix/valiases</tt> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <tt>/etc/postfix/valiases</tt> is changed) * Create <tt>/etc/aliases</tt> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <tt>telnet</tt> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <tt>mail</tt> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# apt-get install amavis-new spamassassin clamav-daemon root@server:~# apt-get install libnet-dns-perl libmail-spf-query-perl pyzor razor root@server:~# apt-get install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <tt>/etc/amavis/conf.d/20-debian-defaults</tt> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <tt>/etc/postfix/main.cf</tt> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <tt>/etc/postfix/master.cf</tt> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <tt>recipient_delimiter</tt> option on Postfix is rather neat. Given a <tt>recipient_delimiter</tt> of '<tt>.</tt>', messages to <tt>user.ext1@domain1.com</tt> are handled in an interesting way. If <tt>user.ext1@domain1.com</tt> is a valid user, the message is put in their mailbox. If <tt>user.ext1@domain1.com</tt> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <tt>user@domain1.com</tt>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <tt>/etc/postfix/main.cf</tt> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <tt>/etc/postfix/header_checks</tt> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <tt>/etc/postfix/main.cf</tt> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] b2331c008e54ff08f8dcc2637f3f2c2380d09df3 2437 2433 2012-10-03T15:37:30Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /home/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 9d2950126abd175c7d50983e2603f3af167b76cf 2438 2437 2012-10-03T16:06:57Z Neil 2 /* Basic installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/ user2@domain1.com domain1.com/user2/ user1@domain2.com domain2.com/user1/ user3@domain2.com domain2.com/user3/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 8669bab68d1a36bd044daf71cd206fcdb3cf6130 2440 2438 2012-10-03T16:10:00Z Neil 2 /* Basic installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] fbcd569118e6cbc138e96e0f8e8cb1a46e8d0c3a 2441 2440 2012-10-03T16:11:09Z Neil 2 /* Adding TLS */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth # smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 66da68c3f6050314666e99011198a40cd5cef476 2442 2441 2012-10-03T17:57:20Z Neil 2 Undo revision 2441 by [[Special:Contributions/Neil|Neil]] ([[User talk:Neil|Talk]]) wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/dovecot.conf</code> with the <code>client</code> section below so that it accepts authentication requests from Postfix. auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } client { path = /var/spool/postfix/private/auth-client mode = 0660 user = postfix group = postfix } } } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth-client smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] fbcd569118e6cbc138e96e0f8e8cb1a46e8d0c3a 2443 2442 2012-10-03T19:06:03Z Neil 2 /* Allowing off-site users with SASL */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject :(i.e. uncomment the lines) You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] c38fa36593d396fdade8fb59dd9f8714d39ac1b6 2444 2443 2012-10-03T19:07:12Z Neil 2 /* Open submission port (587) */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] [[Category:Server setup]] 470dbad3c755c48dbebe521702260d12112f4dea 2445 2444 2012-10-03T19:08:11Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# /etc/init.d/dovecot restart root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 634392cbda50265580779dd156a6462fd4997c0d 2447 2445 2012-10-03T19:37:46Z Neil 2 /* Allowing off-site users with SASL */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# /etc/init.d/spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# /etc/init.d/amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] c70e76bbd8e09dfd1e6654fd0bd96f2af551bcff 2448 2447 2012-10-03T19:38:18Z Neil 2 /* Amavis + ClamAV + SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Restart Postfix root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 9d9fd1f3e9dce45cafbfe0faa92e47f5e22843b4 2449 2448 2012-10-03T19:43:51Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavis-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 1fee5a14631d67d1c79d14a621ad539c806b2b93 2453 2449 2012-10-03T20:19:13Z Neil 2 /* Amavis + ClamAV + SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 :then start SpamAssassin Now start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] f314695309f3c641757d1019508bc3e00f1e783f 0 1532 2434 2263 2012-10-03T15:24:21Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot, Deliver, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/ /user2/ /domain2.com/user1/ /user2/ /domain3.com/user1/ ...and so on. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois :(<tt>whois</tt> is needed in a moment for creating passwords) * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password :(If you want to change a user's password, delete their line from <code>/etc/dovecot/passwd</code> before running the <code>mkdovecotpasswd</code> script again.) * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <code>/etc/dovecot/dovecot.conf</code> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = sieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <code>/etc/postfix/main.cf</code> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <code>/etc/postfix/master.cf</code> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Create an an empty log file for <tt>deliver</tt> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 5443c399c3f8e2ca03d2602210acf6a98b9c82ef 2439 2434 2012-10-03T16:09:36Z Neil 2 /* Virtual user structure */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot, Deliver, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois :(<tt>whois</tt> is needed in a moment for creating passwords) * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password :(If you want to change a user's password, delete their line from <code>/etc/dovecot/passwd</code> before running the <code>mkdovecotpasswd</code> script again.) * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <code>/etc/dovecot/dovecot.conf</code> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = sieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <code>/etc/postfix/main.cf</code> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <code>/etc/postfix/master.cf</code> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Create an an empty log file for <tt>deliver</tt> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] e5c396f90ff8e36177d9380f58d90d6270ae90dc 2446 2439 2012-10-03T19:09:18Z Neil 2 /* Dovecot configuration */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot, Deliver, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Dovecot configuration == * Install Dovecot root@server:~# apt-get install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<tt>whois</tt> is needed in a moment for creating passwords) * Create the <tt>vmail</tt> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Update the <tt>/etc/dovecot/dovecot.conf</tt> file. base_dir = /var/run/dovecot/ protocols = imap imaps log_path = /var/log/dovecot.log info_log_path = /var/log/dovecot.info login_dir = /var/run/dovecot/login login_chroot = yes login_user = dovecot ssl_disable = no ssl_cert_file = /etc/ssl/domain/certs/imap-cert.pem ssl_key_file = /etc/ssl/domain/private/imap-key.pem protocol imap { login_executable = /usr/lib/dovecot/imap-login mail_executable = /usr/lib/dovecot/imap } valid_chroot_dirs = /var/spool/vmail default_mail_env = maildir:/home/vmail/%d/%n disable_plaintext_auth = no auth default { mechanisms = plain userdb passwd-file { args = /etc/dovecot/users } passdb passwd-file { args = /etc/dovecot/passwd } socket listen { master { path = /var/run/dovecot-auth-master mode = 0600 user = vmail group = mail } } } auth_executable = /usr/lib/dovecot/dovecot-auth auth_user = root auth_verbose = yes :note that only the IMAP protocol is supported. Also, I don't really bother with IMAPS, as the [[Firewall setup|firewall]] ensures that the only machines that can connect to Dovecot are on my LAN. If you want to access email remotely via IMAP, you may need to pay more attention to this (see the [[#See also|other resources]] below for details). Personally, I use a [[Webmail setup|webmail]] system for accessing email off-site. == Create virtual users == The Dovecot configuration file above shows that the information on the virtual users is stored in the files <tt>/etc/dovecot/users</tt> and <tt>/etc/dovecot/passwd</tt>. Users are added to these files by the script <tt>/etc/dovecot/adddovecotuser</tt>: #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain::5000:5000::/home/vmail/$domain/$username/:/bin/false::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /home/vmail/$domain/$username 5000:5000 # add the user to the Postfix virtual map file echo $1 $domain/$username >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :and passwords are added with the <tt>/etc/dovecot/mkdovecotpasswd</tt> script #!/bin/bash echo "$1:`mkpasswd --hash=md5 $2`" >> /etc/dovecot/passwd * Make these files executable: root@server:~# chmod a+x /etc/dovecot/adddovecotusers root@server:~# chmod a+x /etc/dovecot/mkdovecotpasswd * To add a user, run root@server:~# /etc/dovecot/adddovecotusers user1@domain1.com * and then create their password root@server:~# /etc/dovecot/mkdovecotpasswd user1@domain1.com password :(If you want to change a user's password, delete their line from <code>/etc/dovecot/passwd</code> before running the <code>mkdovecotpasswd</code> script again.) * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * You now need to ensure that the Dovecot log files are included in the <tt>logrotate</tt>system. Create the file <tt>/etc/logrotate.d/dovecot</tt> /var/log/dovecot*log { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } /var/log/dovecot*info { weekly missingok rotate 52 compress delaycompress notifempty # Note the odd permissions: this is needed because deliver doesn't know what user it runs under create 666 root adm sharedscripts } :This should be picked up automatically the next time <tt>logrotate</tt> is run by <tt>cron</tt>. == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki.dovecot.org/LDA Deliver], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. Note that the implementation of [http://wiki.dovecot.org/LDA/Sieve Sieve in Dovecot] seems to be incomplete, as several rule types (e.g. regexp) don't work. See the [http://wiki.dovecot.org/LDA/Postfix Dovecot page on Postfix] for details. * Modify <code>/etc/dovecot/dovecot.conf</code> to include the configuration for Deliver protocol lda { postmaster_address = postmaster@domain.tld auth_socket_path = /var/run/dovecot-auth-master mail_plugins = sieve log_path = /var/log/dovecot-deliver.log info_log_path = /var/log/dovecot-deliver.log } * Update <code>/etc/postfix/main.cf</code> to enable Dovecot as an LDA dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Update <code>/etc/postfix/master.cf</code> # Dovecot LDA dovecot unix - n n - - pipe # Use this line if you're not using dovecot's 'deliver' LDA # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # use this line if using deliver and you want it to create new maildirs depending on extension # flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} -m ${extension} # use this line if using deliver and arbitrary extensions go into the central inbox flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${user}@${nexthop} * Create an an empty log file for <tt>deliver</tt> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 85b9f7b557edc2fb1f3d54a4e1734a3fb8b7baa9 2450 2446 2012-10-03T20:17:34Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot log files are included in the <code>logrotate</code>system. * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: syslog_facility = local1 * Create <code>/etc/rsyslog.d/20-dovecot.conf</code> to contain: local1.* -/var/log/dovecot.log local1.info -/var/log/dovecot.info local1.warn -/var/log/dovecot.warn local1.err -/var/log/dovecot.err :msg,contains,"stored mail into mailbox"\ -/var/log/dovecot.lda * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log /var/log/dovecot.info /var/log/dovecot.warn /var/log/dovecot.err /var/log/dovecot.lda { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } * Restart <code>rsyslog</code> root@server:~# service rsyslog restart == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 8d61f1013d5d52a09c6a166aebfb20f5ce7432d9 2451 2450 2012-10-03T20:18:04Z Neil 2 /* Initial Dovecot configuration */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot log files are included in the <code>logrotate</code>system. * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: syslog_facility = local1 * Create <code>/etc/rsyslog.d/20-dovecot.conf</code> to contain: local1.* -/var/log/dovecot.log local1.info -/var/log/dovecot.info local1.warn -/var/log/dovecot.warn local1.err -/var/log/dovecot.err :msg,contains,"stored mail into mailbox"\ -/var/log/dovecot.lda * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log /var/log/dovecot.info /var/log/dovecot.warn /var/log/dovecot.err /var/log/dovecot.lda { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } * Restart <code>rsyslog</code> root@server:~# service rsyslog restart == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 81dfe60108a7ab20aa851f95ab559d04f3291bda 2452 2451 2012-10-03T20:18:22Z Neil 2 /* Logging */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot log files are included in the <code>logrotate</code> system. * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: syslog_facility = local1 * Create <code>/etc/rsyslog.d/20-dovecot.conf</code> to contain: local1.* -/var/log/dovecot.log local1.info -/var/log/dovecot.info local1.warn -/var/log/dovecot.warn local1.err -/var/log/dovecot.err :msg,contains,"stored mail into mailbox"\ -/var/log/dovecot.lda * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log /var/log/dovecot.info /var/log/dovecot.warn /var/log/dovecot.err /var/log/dovecot.lda { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } * Restart <code>rsyslog</code> root@server:~# service rsyslog restart == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 3586d10ddfab43042c83fd7432bb36e6e216e6d0 0 1530 2435 1963 2012-10-03T15:25:44Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <tt>/etc/ssl/domain</tt>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create the <tt>/etc/ssl/openssl.cnf</tt> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME = . RANDFILE = $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids # To use this configuration file with the "-extfile" option of the # "openssl x509" utility, name here the section containing the # X.509v3 extensions to use: # extensions = # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca' and 'req'. # Add a simple OID like this: # testoid1=1.2.3.4 # Or use config file substitution like this: # testoid2=${testoid1}.5.6 #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. #unique_subject = no # Set to 'no' to allow creation of # several ctificates with same subject. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/crl.pem # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert # Comment out the following two lines for the "traditional" # (and highly broken) format. name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options # Extension copying option: use with caution. # copy_extensions = copy # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs # so this is commented out by default to leave a V1 CRL. # crlnumber must also be commented out to leave a V1 CRL. # crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering # A few difference way of specifying how similar the request should look # For type CA, the listed attributes must be the same, and the optional # and supplied fields are just that :-) policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy # At this point in time, you must list all acceptable 'object' # types. [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert # Passwords for private keys if not present they will be prompted for # input_password = secret # output_password = secret # This sets a mask for permitted string types. There are several options. # default: PrintableString, T61String, BMPString. # pkix : PrintableString, BMPString. # utf8only: only UTF8Strings. # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). # MASK:XXXX a literal mask value. # WARNING: current versions of Netscape crash on BMPStrings or UTF8Strings # so use this option with caution! string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name # we can do this but it is not needed normally :-) #1.organizationName = Second Organization Name (eg, company) #1.organizationName_default = World Wide Web Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) #organizationalUnitName_default = commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 # SET-ex3 = SET extension number 3 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName [ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] # Extensions for a typical CA # PKIX recommendation. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always # This is what PKIX recommends but some broken software chokes on critical # extensions. #basicConstraints = critical,CA:true # So we do this instead. basicConstraints = CA:true # Key usage: this is typical for a CA certificate. However since it will # prevent it being used as an test self-signed certificate it is best # left out by default. # keyUsage = cRLSign, keyCertSign # Some might want this also # nsCertType = sslCA, emailCA # Include email address in subject alt name: another PKIX recommendation # subjectAltName=email:copy # Copy issuer details # issuerAltName=issuer:copy # DER hex encoding of an extension: beware experts only! # obj=DER:02:03 # Where 'obj' is a standard or added object # You can even override a supported extension: # basicConstraints= critical, DER:30:03:01:01:FF [ crl_ext ] # CRL extensions. # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. # issuerAltName=issuer:copy authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] # These extensions should be added when creating a proxy certificate # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName # This really needs to be in place for it to be a proxy certificate. proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] 96213164803f6ad919db71340d53cd6a5ea73bf0 0 1545 2436 2203 2012-10-03T15:25:49Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server mysql-client == Passwords == The first thing is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <code>host_name</code> is the name of this host, e.g. <code>server</code>. :(seemingly not needed for Ubuntu 10.04) * Add passwords to the <code>root</code> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'127.0.0.1' = PASSWORD('newpwd'); == Remote access == If you want to access MySQL databases from a remote machine, you'll need to have MySQL accept connections across the network. Edit <code>/etc/mysql/my.cnf</code> and change the <code>bind-address</code> entry to be the IP of the machine: bind-address = 192.168.2.252 The restart MySQL: root@server:~# /etc/init.d/mysql restart Note that the [[Firewall setup|firewall]] is configured to only allow connections to MySQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] 99e97693deefb619b34e9efb5c154368da0cd175 0 1531 2454 2453 2012-10-03T20:19:46Z Neil 2 /* SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit <code>/etc/amavis/conf.d/20-debian-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 84730bea811db933454f336f7dd4a4ab0c3b5c59 2502 2454 2013-07-29T13:50:21Z Neil 2 /* Amavis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 60f5d17c2f74ef6b0d25fe05588c4190b2e93588 0 1532 2455 2452 2012-10-04T09:06:52Z Neil 2 /* Logging */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } :(Dovecot recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot*.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 1a5a181b1fa2168e78d588c7f5ee37cc538aa125 2456 2455 2012-10-04T09:07:06Z Neil 2 /* Logging */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } :(Dovecot recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot*.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 3d7097c7d3506a0329eaaa5d910e77836db92f61 2457 2456 2012-10-04T09:14:02Z Neil 2 /* Logging */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot*.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] ec10e5c0fe1478a3d0cb8e533d7724ac02c2d2bc 2467 2457 2012-10-04T18:22:46Z Neil 2 /* Troubleshooting */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot*.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 4bd6929e0c78e0dec34f2ef10062a7181a3a2901 2480 2467 2012-10-15T14:52:19Z Neil 2 /* Logging */ Added LDA log wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.* { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 3e946858d010481ecfe4dafc1ecf18ff4712e967 2481 2480 2012-10-15T14:53:18Z Neil 2 /* Logging */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.* { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 36d159aa31b237f687a78274d23387c00a32fb0e 2499 2481 2013-04-19T12:22:31Z Neil 2 /* Logging */ Fixed bug in logrotate script wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 32b152436223b7ddf0105c9c8d7f418dbb643c28 0 1534 2458 2426 2012-10-04T09:21:42Z Neil 2 /* Upgrading from previous versions */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/upload> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/upload"; ### $wgUploadDirectory = "$IP/upload"; ### == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/upload/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/upload"; $wgUploadDirectory = "$IP/upload"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 120d8e7cf5c1f27a1943b71a0623d3827363a6c6 2459 2458 2012-10-04T09:25:23Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/Test.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/images"; ### $wgUploadDirectory = "$IP/images"; ### == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] bba4c07ef72a36a9f23751386ba9a4613a81326d 2460 2459 2012-10-04T10:09:41Z Neil 2 /* Creating the farm */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; $wgUploadPath = "$wgScriptPath/images"; ### $wgUploadDirectory = "$IP/images"; ### == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 2eeed495b35c96ea422c65e0db6be46750a68fb4 2461 2460 2012-10-04T11:25:34Z Neil 2 /* Creating the farm */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### # If PHP's memory limit is very low, some operations may fail. #ini_set( 'memory_limit', '20M' ); ini_set( 'memory_limit', '128M' ); ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 355aec5ed61d3c83cccf6c01bdfd68cca4269838 2462 2461 2012-10-04T11:26:54Z Neil 2 /* Creating the farm */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### # If PHP's memory limit is very low, some operations may fail. #ini_set( 'memory_limit', '20M' ); ini_set( 'memory_limit', '128M' ); ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "$wgUploadPath/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 128e6bc0b3643b392df5ee0ac5fc12e729056b62 2463 2462 2012-10-04T11:43:26Z Neil 2 /* Adding a logo */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### # If PHP's memory limit is very low, some operations may fail. #ini_set( 'memory_limit', '20M' ); ini_set( 'memory_limit', '128M' ); ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs $wgArticlePath = "/$1"; ### # $wgArticlePath = "$wgScript?title=$1"; $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "/mediawiki/images/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 3efaeb9dbcc74ab227e3bbe38b5ff9b7e090a7d4 0 1535 2464 2179 2012-10-04T12:35:38Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I use [http://www.squirrelmail.org/index.php SquirrelMail] as a webmail interface. To install Squirrelmail, install the packages: root@server:~# aptitude install squirrelmail ibritish wbritish (which also includes a British English dictionary). SquirrelMail has a very nice configuration script that does just about all the work for you, given that the [[Dovecot server setup|IMAP server]] and [[Web server setup|Web server]] are already working. Simply run the script: root@server:~# squirrelmail-configure select 'D' to configure using Dovecot, and follow the instructions. As SquirrelMail, Apache, and Dovecot are all running on the same machine, there's no need to use IMAPS to secure the communication between them. Next, you need to modify the <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> file to point to SquirrelMail. Adjust the <code>DocumentRoot</code> setting in the secure section of that file to DocumentRoot /usr/share/squirrelmail Restart the server: root@server:~# /etc/init.d/apache2 restart And you should have a SquirrelMail webmail site running! Check the configuration file by browsing to <nowiki>https://your-squirrelmail-location/src/configtest.php</nowiki>. == Change the login logo == If you want to change the logo on the login screen, replace the file <code>/usr/share/squirrelmail/images/sm_logo.png</code> with another image, making sure it's the same width and height as the original. The site's favicon is <code>/usr/share/squirrelmail/favicon.ico</code>. == Updating and migrating == To migrate from a previous installation, copy the files in <code>/var/lib/squirrelmail/data</code> to keep users' preferences. == See also == * [http://www.squirrelmail.org/documentation/ SquirrelMail documentation] * [https://help.ubuntu.com/community/Squirrelmail Hints for SquirrelMail on Ubuntu] [[Category:Server setup]] 1d6565b50096f2ba5fc7397ca6a3a64b9e49f822 0 1533 2465 2266 2012-10-04T12:40:18Z Neil 2 /* Setting a default virtual host */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# service apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <tt>/etc/apache2/ssl/certs</tt>. Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. Now, create the configuration file for the <tt>squirrelmail.domain.tld</tt> site, <tt>/etc/apache2/sites-available/squirrelmail.domain.tld</tt> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/certs/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] b4f5c3fda66122a59dcf2cfcd38ebd033ec0177f 2466 2465 2012-10-04T13:04:06Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# service apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <cocde>/etc/apache2/ssl/certs</code>. root@server:~# mkdir -p /etc/apache2/ssl/private root@server:~# mkdir -p /etc/apache2/ssl/certs root@server:~# cp /etc/ssl/njae/certs/squirrelmail-cert.pem /etc/apache2/ssl/certs/ root@server:~# cp /etc/ssl/njae/private/squirrelmail-key.insecure.pem /etc/apache2/ssl/private/ Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. root@server:~# chown -R www-data:www-data /etc/apache2/ssl/ root@server:~# chmod -R 640 /etc/apache2/ssl/ Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/private/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 39f1354702645c24de807d4eb13eb63bd2e2c3d5 0 1542 2468 2243 2012-10-05T13:56:25Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# /etc/init.d/cupsys restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is only supported in HPLIP 2.8.10 and higher, but Ubuntu 8.04 only has HPLIP 2.8.2. So I need to update it. NB: Update the software before attaching the printer. * Remove the existing HPLIP installation. root@desktop:~# apt-get purge hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions] That's it! == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] bdbd3f60984838c1363e103da75128d9d7c66f70 2469 2468 2012-10-05T13:57:36Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is only supported in HPLIP 2.8.10 and higher, but Ubuntu 8.04 only has HPLIP 2.8.2. So I need to update it. NB: Update the software before attaching the printer. * Remove the existing HPLIP installation. root@desktop:~# aptitude purge hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions] That's it! == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] bcdf244fa4fe01c8e8f6b77e30fe40d2b1f973c6 2470 2469 2012-10-05T14:30:47Z Neil 2 /* Updating HPLIP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is only supported in HPLIP 2.8.10 and higher, but Ubuntu 8.04 only has HPLIP 2.8.2. So I need to update it. NB: Update the software before attaching the printer. * Don't remove any existing HPLIP installation, as you need it for libraries used by the installer. If necessary, add these packages: root@desktop:~# aptitude install hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs printer-driver-postscript-hp * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions]. * If the plugin installation fails cancel it in the installer and run it again with root permissions: root@desktop:~# hp-plugin-ubuntu That's it! == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] ea83cb8aac1af41f73563e6ae27fef5346b49b06 2471 2470 2012-10-05T14:33:29Z Neil 2 /* Updating HPLIP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it. NB: Update the software before attaching the printer. * Don't remove any existing HPLIP installation, as you need it for libraries used by the installer. If necessary, add these packages: root@desktop:~# aptitude install hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs printer-driver-postscript-hp * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions]. * If the plugin installation fails cancel it in the installer and run it again with root permissions: root@desktop:~# hp-plugin-ubuntu That's it! == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] 8851a116773c991c9e4446911c74b0ba40b0b865 2474 2471 2012-10-05T14:41:39Z Neil 2 /* Updating HPLIP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it, as described on the [[Scanner setup]] page. == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] eb189ff7b0798d1bdf74f4dc76d9cdedc86a7d2c 2497 2474 2013-03-03T13:02:52Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it, as described on the [[Scanner setup]] page. ==Bugfix== CUPS on Ubuntu 12.04 has a [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/872483 bug that affects the USB backend], preventing the printer printing more than one job with it needing resetting. Once the printer is set up, issue the command: root@server:~# lpadmin -p <printer> -o usb-no-reattach-default=true == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] 895b5af042750808312815503e646e576eb85340 0 1563 2472 2219 2012-10-05T14:40:33Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} == Install HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it. NB: Update the software before attaching the printer. * Don't remove any existing HPLIP installation, as you need it for libraries used by the installer. If necessary, add these packages: root@desktop:~# aptitude install hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs printer-driver-postscript-hp * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions]. * If the plugin installation fails cancel it in the installer and run it again with root permissions: root@desktop:~# hp-plugin-ubuntu That's it! (See also the [[CUPS server setup|printer setup]] page) == Server setup (untested) == (We're using the desktop machine as the server for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# aptitude install xsane xsane-doc sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [http://ubuntuforums.org/showthread.php?p=4916614 this forum post] and [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# service xinetd restart * Open the right port in the [[Firewall setup|firewall]] by adding these lines to <code>/etc/iptables.rules</code>. Make sure the scanner is only available to machines on this LAN. ## Network scanning (port 6566) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport saned -j ACCEPT :and restart the firewall root@desktop:~# /etc/init.d/iptables restart * As this is a multi-function printer, restart CUPS and check that it can still print to the printer. == Linux client setup (untested) == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# aptitude install sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup (untested) == Use [http://sanetwain.ozuzo.net/ SaneTwain] and follow the instructions. Unfortunately, I couldn't get the scanner working reliably across a network at all, so SaneTwain didn't have a hope of working. == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] 49010949c986af1adaa450e2ba1e3f3290a57bb9 2473 2472 2012-10-05T14:40:50Z Neil 2 /* Install HPLIP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Subversion setup|Subversion]] |} == Install HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it. NB: Update the software before attaching the printer. * Don't remove any existing HPLIP installation, as you need it for libraries used by the installer. If necessary, add these packages: root@desktop:~# aptitude install hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs printer-driver-postscript-hp * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions]. * If the plugin installation fails, cancel it in the installer and run it again with root permissions: root@desktop:~# hp-plugin-ubuntu That's it! (See also the [[CUPS server setup|printer setup]] page) == Server setup (untested) == (We're using the desktop machine as the server for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# aptitude install xsane xsane-doc sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [http://ubuntuforums.org/showthread.php?p=4916614 this forum post] and [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# service xinetd restart * Open the right port in the [[Firewall setup|firewall]] by adding these lines to <code>/etc/iptables.rules</code>. Make sure the scanner is only available to machines on this LAN. ## Network scanning (port 6566) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport saned -j ACCEPT :and restart the firewall root@desktop:~# /etc/init.d/iptables restart * As this is a multi-function printer, restart CUPS and check that it can still print to the printer. == Linux client setup (untested) == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# aptitude install sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup (untested) == Use [http://sanetwain.ozuzo.net/ SaneTwain] and follow the instructions. Unfortunately, I couldn't get the scanner working reliably across a network at all, so SaneTwain didn't have a hope of working. == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] f5a1a85111513567360962dd5aa9ac4372c924a8 2484 2473 2012-10-19T13:59:17Z Neil 2 Removed links to Subversion wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[CUPS server setup|Print server]] |align="center" width="50%"| [[Git setup|Git]] |} == Install HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it. NB: Update the software before attaching the printer. * Don't remove any existing HPLIP installation, as you need it for libraries used by the installer. If necessary, add these packages: root@desktop:~# aptitude install hplip hplip-gui hpijs hpijs-ppds foomatic-db-hpijs printer-driver-postscript-hp * Download the [http://hplipopensource.com/hplip-web/gethplip.html up-to-date HPLIP] * Follow the [http://hplipopensource.com/hplip-web/install/install/index.html automatic installation instructions]. * If the plugin installation fails, cancel it in the installer and run it again with root permissions: root@desktop:~# hp-plugin-ubuntu That's it! (See also the [[CUPS server setup|printer setup]] page) == Server setup (untested) == (We're using the desktop machine as the server for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@desktop:~# aptitude install xsane xsane-doc sane-utils xinetd * No additional work should be needed for doing the scanner under the first-created user. Check that this user can access the scanner, either with xsane or scanimage: user@desktop:~# scanimage -L Making the scanner available to the rest of the network takes a bit more effort, mainly because scanner device identification is broken in Ubuntu 8.04. * Find the manufacturer and device IDs for your scanner root@desktop:~# lsusb :You'll get a line something like Bus 005 Device 004: ID 03f0:4e17 Hewlett-Packard :In the ID field, the first number (03f0) is the vendor's code and the second (4e17) is the device code. * Create the file <code>/etc/udev/rules.d/41-scanner.rules</code> # Hewlett-Packard Color LaserJet CM1312 MFP SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", \ ATTRS{idVendor}=="03f0", ATTRS{idProduct}=="4e17", \ MODE:="0664", GROUP:="scanner" :(as described in [http://ubuntuforums.org/showthread.php?p=4916614 this forum post] and [https://bugs.launchpad.net/ubuntu/+source/udev/+bug/210421 this bug report]). Replace the vendor and product codes with the ones you found above. Note that the rules are case sensitive and the colons in the assignments are required. * Restart <code>udev</code> root@desktop:~# /etc/init.d/udev restart * Add users who want to use the scanner to the <code>scanner</code> group. In particular, add the <code>saned</code> user. root@desktop:~# adduser saned scanner * Test that users can access the scanner root@desktop:~# sudo -u saned scanimage -L device `hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one * Create the file <code>/etc/xinetd.d/saned</code> so that xinetd knows about the saned daemon service saned { socket_type = stream server = /usr/sbin/saned protocol = tcp user = saned group = scanner wait = no disable = no } * Add the following line to <code>/etc/sane.d/saned.conf</code> to share the printer with all computers on your subnet: 192.168.1.0/24 * Restart xinetd root@desktop:~# service xinetd restart * Open the right port in the [[Firewall setup|firewall]] by adding these lines to <code>/etc/iptables.rules</code>. Make sure the scanner is only available to machines on this LAN. ## Network scanning (port 6566) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport saned -j ACCEPT :and restart the firewall root@desktop:~# /etc/init.d/iptables restart * As this is a multi-function printer, restart CUPS and check that it can still print to the printer. == Linux client setup (untested) == (We're using the server machine as the client for the scanning service.) * Add xsane and sane-utils and xinetd packages (sane-utils and xinetd only for non-GUI machines) root@server:~# aptitude install sane-utils xinetd * Add the server name or IP address of the scanner server to <code>/etc/sane.d/net.conf</code> 192.168.1.251 * Test the users can access the scanner root@server:~# scanimage -L device `net:desktop.domain.tld:hpaio:/usb/HP_Color_LaserJet_CM1312_MFP?serial=00XXX886XXX2' is a Hewlett-Packard HP_Color_LaserJet_CM1312_MFP all-in-one == Windows client setup (untested) == Use [http://sanetwain.ozuzo.net/ SaneTwain] and follow the instructions. Unfortunately, I couldn't get the scanner working reliably across a network at all, so SaneTwain didn't have a hope of working. == See also == * [https://help.ubuntu.com/community/ScanningHowTo Ubuntu scanner documentation] * [http://penguin-breeder.org/sane/saned/ SaneD instructions] f0f592b44cba09a5f0ea4e5e407fcdc577519c4e 0 1594 2475 2176 2012-10-06T18:43:39Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 * Add a line to the <code>/etc/cron.d/awstats</code> file, the same as the lines for the web sites: 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=mail -update >/dev/null == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 4f71357b17b8c4f20fb506e8585a4312e07ba14d 2476 2475 2012-10-07T13:20:08Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * Set up a cron job to update the statistics periodically. Add lines to <code>/etc/cron.d/awstats</code> for each site you want stats for: # 0,10,20,30,40,50 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=awstats -update >/dev/null 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.domain.tld/awstats.pl ] && /var/www/cgi-bin.domain.tld/awstats.pl -config=site2.domain.tld -update >/dev/null :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 * Add a line to the <code>/etc/cron.d/awstats</code> file, the same as the lines for the web sites: 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=mail -update >/dev/null ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 6741ae7d736360a355bad776588d72c4d50c9129 2477 2476 2012-10-07T16:25:07Z Neil 2 /* Websites */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 * Add a line to the <code>/etc/cron.d/awstats</code> file, the same as the lines for the web sites: 0,10,20,30,40,50 * * * * root [ -x /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl ] && /var/www/cgi-bin.scripts.njae.me.uk/awstats.pl -config=mail -update >/dev/null ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] ca8f2a2c56c97038e79970fe55cfc5abdbb2d03c 2478 2477 2012-10-07T16:25:23Z Neil 2 /* Mail servers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# /etc/init.d/apache2/reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 090f67214a91f5606f0c53085cfffc6addecf4f4 2479 2478 2012-10-07T16:25:48Z Neil 2 /* Apache configuration */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# service apache2 reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 74775b19a0313619f40de6e975cee65fc84aa849 0 1593 2482 2293 2012-10-19T13:56:27Z Neil 2 Changed to use gitolite, not gitosis wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Subversion setup|Subversion]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; Add Apache's user to the git group: root@server:~# usermod -a -G git www-data Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. Reload apache root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 95169fa6ea17bdd22cc70327ecc233f839ad07e9 2483 2482 2012-10-19T13:59:11Z Neil 2 Removed links to Subversion wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; Add Apache's user to the git group: root@server:~# usermod -a -G git www-data Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. Reload apache root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 7dac64054f59096bd5ef9e4508bb390e46fe4165 2487 2483 2012-10-19T14:10:29Z Neil 2 /* Setup gitweb */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 27d0fe45d0dd112d6596ce70937241d5f3d8bcc9 2488 2487 2012-10-19T14:14:01Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] ce62f7f66c85bf8c6bbe158cf01a7a05845775dc 2501 2488 2013-05-26T19:34:00Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== To clone a repo: git clone git@git.domain.tld:repo.git ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 62493e5b0e8d68b291e644efefa0a9b9faf6ef90 2503 2501 2013-10-21T17:53:23Z Neil 2 Added git-daemon section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create it in <code>gitolite-admin</code> (and push the changes), then clone it on the client machine. ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] e55e9fbc0c3379b4f9a2fe22ac1cd7f28093513e 0 1515 2485 2432 2012-10-19T13:59:44Z Neil 2 Removed links to Subversion wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 10.04 LTS (adapted from 8.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 3bde84425f962aa50e5ef4b171b6c1f2b885b8ef 2490 2485 2012-11-30T13:50:46Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 10.04 LTS (adapted from 8.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] dbac577d5290758848198fe8c9b6838b5fbddb29 0 1526 2486 2423 2012-10-19T14:06:37Z Neil 2 /* Clean up Apt cache */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * texlive-latex-base * texlive-latex-extra * texlive-science * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 34a330f77af87d96f18b25a3aa229b3d17f69a28 2492 2486 2012-11-30T13:52:35Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * texlive-latex-base * texlive-latex-extra * texlive-science * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 4e5134f765a27e288eec65bfaf353ee7469639bd 2498 2492 2013-03-03T13:21:38Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde * edubuntu-addon-young * edubuntu-addon-science * education-logic-games * gnome-games * kdegames * gnubg * gimp * gimp-data-extras * gimp-python * gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer * mplayer-fonts * mencoder * xine-ui * xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk * sun-java6-jre * sun-java6-plugin * sun-java6-bin * sun-java6-javadb * sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * ruby-full * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * jabref * kbibtex * texlive-latex-base * texlive-latex-extra * texlive-science * mmv * kdesvn * git-core * git-docs * git-svn * git-email * git-gui * gitk * tofrodos * ttf-mscorefonts-installer * curl === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 8cc217246c1ed31caad5fef2026e1512c1305256 2500 2498 2013-04-29T14:56:34Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * ghc6 (Haskell) * ghc6-prof * haskell-mode * python-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] cfc3e9fde284cbf4d24791de65847d5b85eb10e1 0 1629 2489 2012-11-30T13:49:14Z Neil 2 Created page with "I use VNC for remote GUI access to my Raspberry Pi. As I only run it across my LAN, I have no need to encrypt VNC traffic. If you're accessing the RPi from other locations, you'l…" wikitext text/x-wiki I use VNC for remote GUI access to my Raspberry Pi. As I only run it across my LAN, I have no need to encrypt VNC traffic. If you're accessing the RPi from other locations, you'll want to use either SSH tunnelling or VPN to get secure access. ==VNC Server on Raspberry Pi== This makes the RPi's desktop available on port 5901 (desktop :1). It creates only a single desktop. If multiple clients connect, they get to fight over who has control! * Install TightVNC server pi@raspberrypi ~ $ sudo aptitude install tightvncserver * Start the server to set a password for the session pi@raspberrypi ~ $ tightvncserver :1 :Set the password when asked, then kill that session for further updates. pi@raspberrypi ~ $ tightvncserver -kill :1 * Create the file <code>/etc/init.d/tightvncserver</code> (use <code>sudo</code>, as only root has write permission to that directory): #!/bin/sh ### BEGIN INIT INFO # Provides: tightvncserver # Required-Start: $local_fs # Required-Stop: $local_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start/stop tightvncserver ### END INIT INFO # More details see: # http://www.penguintutor.com/linux/tightvnc ### Customize this entry # Set the USER variable to the name of the user to start tightvncserver under export USER='pi' ### End customization required eval cd ~$USER case "$1" in start) su $USER -c '/usr/bin/tightvncserver :1 -geometry 1200x674 -depth 16 -pixelformat rgb565' echo "Starting TightVNC server for $USER " ;; stop) pkill Xtightvnc echo "Tightvncserver stopped" ;; *) echo "Usage: /etc/init.d/tightvncserver {start|stop}" exit 1 ;; esac exit 0 :Change the <code>geometry</code> resolution to suit. 1200×674 fits nicely on my laptop's screen. * Make the script executable and run on boot: pi@raspberrypi ~ $ sudo a+x /etc/init.d/tightvncserver pi@raspberrypi ~ $ sudo update-rc.d tightvncserver defaults pi@raspberrypi ~ $ sudo service tightvncserver start ==VNC Client on Linux== Simplicity itself. user@desktop:~$ sudo aptitude install xtightvncviewer user@:desktop~$ vncviewer 192.168.2.11:1 (where 192.168.2.11 is the IP number of my RPi. Yours will differ.) ==VNC client on Android== There are many VNC clients. I use [https://play.google.com/store/apps/details?id=android.androidVNC&feature=search_result#?t=W251bGwsMSwxLDEsImFuZHJvaWQuYW5kcm9pZFZOQyJd android-vnc-viewer]. The more popular alternative is [https://play.google.com/store/apps/details?id=com.wyse.pocketcloudfree&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS53eXNlLnBvY2tldGNsb3VkZnJlZSJd PocketCloud Remote RDP / VNC] but the interface for that still requires a Menu button, which disappeared with Android 4.0. The VNC client should connect to your RPi's IP number using either desktop number :1 or port 5901. ==See also== [[Category:Server setup]] dc30d376864e3534c70c426b016e507f74870b89 2493 2489 2012-11-30T13:52:39Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} I use VNC for remote GUI access to my Raspberry Pi. As I only run it across my LAN, I have no need to encrypt VNC traffic. If you're accessing the RPi from other locations, you'll want to use either SSH tunnelling or VPN to get secure access. ==VNC Server on Raspberry Pi== This makes the RPi's desktop available on port 5901 (desktop :1). It creates only a single desktop. If multiple clients connect, they get to fight over who has control! * Install TightVNC server pi@raspberrypi ~ $ sudo aptitude install tightvncserver * Start the server to set a password for the session pi@raspberrypi ~ $ tightvncserver :1 :Set the password when asked, then kill that session for further updates. pi@raspberrypi ~ $ tightvncserver -kill :1 * Create the file <code>/etc/init.d/tightvncserver</code> (use <code>sudo</code>, as only root has write permission to that directory): #!/bin/sh ### BEGIN INIT INFO # Provides: tightvncserver # Required-Start: $local_fs # Required-Stop: $local_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start/stop tightvncserver ### END INIT INFO # More details see: # http://www.penguintutor.com/linux/tightvnc ### Customize this entry # Set the USER variable to the name of the user to start tightvncserver under export USER='pi' ### End customization required eval cd ~$USER case "$1" in start) su $USER -c '/usr/bin/tightvncserver :1 -geometry 1200x674 -depth 16 -pixelformat rgb565' echo "Starting TightVNC server for $USER " ;; stop) pkill Xtightvnc echo "Tightvncserver stopped" ;; *) echo "Usage: /etc/init.d/tightvncserver {start|stop}" exit 1 ;; esac exit 0 :Change the <code>geometry</code> resolution to suit. 1200×674 fits nicely on my laptop's screen. * Make the script executable and run on boot: pi@raspberrypi ~ $ sudo a+x /etc/init.d/tightvncserver pi@raspberrypi ~ $ sudo update-rc.d tightvncserver defaults pi@raspberrypi ~ $ sudo service tightvncserver start ==VNC Client on Linux== Simplicity itself. user@desktop:~$ sudo aptitude install xtightvncviewer user@:desktop~$ vncviewer 192.168.2.11:1 (where 192.168.2.11 is the IP number of my RPi. Yours will differ.) ==VNC client on Android== There are many VNC clients. I use [https://play.google.com/store/apps/details?id=android.androidVNC&feature=search_result#?t=W251bGwsMSwxLDEsImFuZHJvaWQuYW5kcm9pZFZOQyJd android-vnc-viewer]. The more popular alternative is [https://play.google.com/store/apps/details?id=com.wyse.pocketcloudfree&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS53eXNlLnBvY2tldGNsb3VkZnJlZSJd PocketCloud Remote RDP / VNC] but the interface for that still requires a Menu button, which disappeared with Android 4.0. The VNC client should connect to your RPi's IP number using either desktop number :1 or port 5901. ==See also== [[Category:Server setup]] 9422e9d113c00c75db1620906941e66733296ff6 2494 2493 2012-11-30T13:57:04Z Neil 2 /* See also */ Added links wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} I use VNC for remote GUI access to my Raspberry Pi. As I only run it across my LAN, I have no need to encrypt VNC traffic. If you're accessing the RPi from other locations, you'll want to use either SSH tunnelling or VPN to get secure access. ==VNC Server on Raspberry Pi== This makes the RPi's desktop available on port 5901 (desktop :1). It creates only a single desktop. If multiple clients connect, they get to fight over who has control! * Install TightVNC server pi@raspberrypi ~ $ sudo aptitude install tightvncserver * Start the server to set a password for the session pi@raspberrypi ~ $ tightvncserver :1 :Set the password when asked, then kill that session for further updates. pi@raspberrypi ~ $ tightvncserver -kill :1 * Create the file <code>/etc/init.d/tightvncserver</code> (use <code>sudo</code>, as only root has write permission to that directory): #!/bin/sh ### BEGIN INIT INFO # Provides: tightvncserver # Required-Start: $local_fs # Required-Stop: $local_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start/stop tightvncserver ### END INIT INFO # More details see: # http://www.penguintutor.com/linux/tightvnc ### Customize this entry # Set the USER variable to the name of the user to start tightvncserver under export USER='pi' ### End customization required eval cd ~$USER case "$1" in start) su $USER -c '/usr/bin/tightvncserver :1 -geometry 1200x674 -depth 16 -pixelformat rgb565' echo "Starting TightVNC server for $USER " ;; stop) pkill Xtightvnc echo "Tightvncserver stopped" ;; *) echo "Usage: /etc/init.d/tightvncserver {start|stop}" exit 1 ;; esac exit 0 :Change the <code>geometry</code> resolution to suit. 1200×674 fits nicely on my laptop's screen. * Make the script executable and run on boot: pi@raspberrypi ~ $ sudo a+x /etc/init.d/tightvncserver pi@raspberrypi ~ $ sudo update-rc.d tightvncserver defaults pi@raspberrypi ~ $ sudo service tightvncserver start ==VNC Client on Linux== Simplicity itself. user@desktop:~$ sudo aptitude install xtightvncviewer user@:desktop~$ vncviewer 192.168.2.11:1 (where 192.168.2.11 is the IP number of my RPi. Yours will differ.) ==VNC client on Android== There are many VNC clients. I use [https://play.google.com/store/apps/details?id=android.androidVNC&feature=search_result#?t=W251bGwsMSwxLDEsImFuZHJvaWQuYW5kcm9pZFZOQyJd android-vnc-viewer]. The more popular alternative is [https://play.google.com/store/apps/details?id=com.wyse.pocketcloudfree&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS53eXNlLnBvY2tldGNsb3VkZnJlZSJd PocketCloud Remote RDP / VNC] but the interface for that still requires a Menu button, which disappeared with Android 4.0. The VNC client should connect to your RPi's IP number using either desktop number :1 or port 5901. ==See also== * [http://www.penguintutor.com/linux/tightvnc Penguin Tutor's VNC tutorial] for most of the setup. * [http://www.raspberrypi.org/phpBB3/viewtopic.php?f=63&t=9118&p=128842 an RPi thread on TightVNC] for the extra display settings. * [http://news.metaparadigma.de/linux-setting-up-a-debian-vnc-server-237/ Setting Up a Debian VNC Server (via SSH tunnel)] for a different, general howto, connecting to Gnome via SSH. [[Category:Server setup]] 466f89959d49b08848241598462d50b0e3554664 2495 2494 2012-11-30T13:58:15Z Neil 2 /* VNC Client on Linux */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} I use VNC for remote GUI access to my Raspberry Pi. As I only run it across my LAN, I have no need to encrypt VNC traffic. If you're accessing the RPi from other locations, you'll want to use either SSH tunnelling or VPN to get secure access. ==VNC Server on Raspberry Pi== This makes the RPi's desktop available on port 5901 (desktop :1). It creates only a single desktop. If multiple clients connect, they get to fight over who has control! * Install TightVNC server pi@raspberrypi ~ $ sudo aptitude install tightvncserver * Start the server to set a password for the session pi@raspberrypi ~ $ tightvncserver :1 :Set the password when asked, then kill that session for further updates. pi@raspberrypi ~ $ tightvncserver -kill :1 * Create the file <code>/etc/init.d/tightvncserver</code> (use <code>sudo</code>, as only root has write permission to that directory): #!/bin/sh ### BEGIN INIT INFO # Provides: tightvncserver # Required-Start: $local_fs # Required-Stop: $local_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start/stop tightvncserver ### END INIT INFO # More details see: # http://www.penguintutor.com/linux/tightvnc ### Customize this entry # Set the USER variable to the name of the user to start tightvncserver under export USER='pi' ### End customization required eval cd ~$USER case "$1" in start) su $USER -c '/usr/bin/tightvncserver :1 -geometry 1200x674 -depth 16 -pixelformat rgb565' echo "Starting TightVNC server for $USER " ;; stop) pkill Xtightvnc echo "Tightvncserver stopped" ;; *) echo "Usage: /etc/init.d/tightvncserver {start|stop}" exit 1 ;; esac exit 0 :Change the <code>geometry</code> resolution to suit. 1200×674 fits nicely on my laptop's screen. * Make the script executable and run on boot: pi@raspberrypi ~ $ sudo a+x /etc/init.d/tightvncserver pi@raspberrypi ~ $ sudo update-rc.d tightvncserver defaults pi@raspberrypi ~ $ sudo service tightvncserver start ==VNC Client on Linux== Simplicity itself. user@desktop:~$ sudo aptitude install xtightvncviewer user@desktop:~$ vncviewer 192.168.2.11:1 (where 192.168.2.11 is the IP number of my RPi. Yours will differ.) ==VNC client on Android== There are many VNC clients. I use [https://play.google.com/store/apps/details?id=android.androidVNC&feature=search_result#?t=W251bGwsMSwxLDEsImFuZHJvaWQuYW5kcm9pZFZOQyJd android-vnc-viewer]. The more popular alternative is [https://play.google.com/store/apps/details?id=com.wyse.pocketcloudfree&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS53eXNlLnBvY2tldGNsb3VkZnJlZSJd PocketCloud Remote RDP / VNC] but the interface for that still requires a Menu button, which disappeared with Android 4.0. The VNC client should connect to your RPi's IP number using either desktop number :1 or port 5901. ==See also== * [http://www.penguintutor.com/linux/tightvnc Penguin Tutor's VNC tutorial] for most of the setup. * [http://www.raspberrypi.org/phpBB3/viewtopic.php?f=63&t=9118&p=128842 an RPi thread on TightVNC] for the extra display settings. * [http://news.metaparadigma.de/linux-setting-up-a-debian-vnc-server-237/ Setting Up a Debian VNC Server (via SSH tunnel)] for a different, general howto, connecting to Gnome via SSH. [[Category:Server setup]] 62947ac93851a6a808f3ce22c399fb0b5c333b30 2496 2495 2012-12-13T08:43:03Z Neil 2 /* VNC Server on Raspberry Pi */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |align="center" width="50%"| [[Cryptography setup|Cryptography]] |} I use VNC for remote GUI access to my Raspberry Pi. As I only run it across my LAN, I have no need to encrypt VNC traffic. If you're accessing the RPi from other locations, you'll want to use either SSH tunnelling or VPN to get secure access. ==VNC Server on Raspberry Pi== This makes the RPi's desktop available on port 5901 (desktop :1). It creates only a single desktop. If multiple clients connect, they get to fight over who has control! * Install TightVNC server pi@raspberrypi ~ $ sudo aptitude install tightvncserver * Start the server to set a password for the session pi@raspberrypi ~ $ tightvncserver :1 :Set the password when asked, then kill that session for further updates. pi@raspberrypi ~ $ tightvncserver -kill :1 * Create the file <code>/etc/init.d/tightvncserver</code> (use <code>sudo</code>, as only root has write permission to that directory): #!/bin/sh ### BEGIN INIT INFO # Provides: tightvncserver # Required-Start: $local_fs # Required-Stop: $local_fs # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start/stop tightvncserver ### END INIT INFO # More details see: # http://www.penguintutor.com/linux/tightvnc ### Customize this entry # Set the USER variable to the name of the user to start tightvncserver under export USER='pi' ### End customization required eval cd ~$USER case "$1" in start) su $USER -c '/usr/bin/tightvncserver :1 -geometry 1200x674 -depth 16 -pixelformat rgb565' echo "Starting TightVNC server for $USER " ;; stop) pkill Xtightvnc echo "Tightvncserver stopped" ;; *) echo "Usage: /etc/init.d/tightvncserver {start|stop}" exit 1 ;; esac exit 0 :Change the <code>geometry</code> resolution to suit. 1200×674 fits nicely on my laptop's screen. * Make the script executable and run on boot: pi@raspberrypi ~ $ sudo chmod a+x /etc/init.d/tightvncserver pi@raspberrypi ~ $ sudo update-rc.d tightvncserver defaults pi@raspberrypi ~ $ sudo service tightvncserver start ==VNC Client on Linux== Simplicity itself. user@desktop:~$ sudo aptitude install xtightvncviewer user@desktop:~$ vncviewer 192.168.2.11:1 (where 192.168.2.11 is the IP number of my RPi. Yours will differ.) ==VNC client on Android== There are many VNC clients. I use [https://play.google.com/store/apps/details?id=android.androidVNC&feature=search_result#?t=W251bGwsMSwxLDEsImFuZHJvaWQuYW5kcm9pZFZOQyJd android-vnc-viewer]. The more popular alternative is [https://play.google.com/store/apps/details?id=com.wyse.pocketcloudfree&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS53eXNlLnBvY2tldGNsb3VkZnJlZSJd PocketCloud Remote RDP / VNC] but the interface for that still requires a Menu button, which disappeared with Android 4.0. The VNC client should connect to your RPi's IP number using either desktop number :1 or port 5901. ==See also== * [http://www.penguintutor.com/linux/tightvnc Penguin Tutor's VNC tutorial] for most of the setup. * [http://www.raspberrypi.org/phpBB3/viewtopic.php?f=63&t=9118&p=128842 an RPi thread on TightVNC] for the extra display settings. * [http://news.metaparadigma.de/linux-setting-up-a-debian-vnc-server-237/ Setting Up a Debian VNC Server (via SSH tunnel)] for a different, general howto, connecting to Gnome via SSH. [[Category:Server setup]] e0194db245aa5fd8b318e9519234b51ada8a9d61 0 1587 2491 2162 2012-11-30T13:52:30Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[VNC setup|VNC]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. == Import a secret/public key pair from another source == === Add another identity to a key === === Publish a public key === Keys should be exported to and imported from the [http://wwwkeys.eu.pgp.net/ EU PGP key server]. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == There is a plugin for this, but it hasn't been updated in a long time and no longer works. == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] af23023104d958eea4a98d59cdc72a9ee8d2ce88 0 1593 2504 2503 2013-10-21T17:54:42Z Neil 2 /* Install and setup git-daemon */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create it in <code>gitolite-admin</code> (and push the changes), then clone it on the client machine. ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 586ef0a4d805e4387c2f7ed0aa572b46eccbbb49 0 1 2505 2416 2013-11-06T20:55:49Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == {| align="right" {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://alpha.app.net/neilnjae @neilnjae] on App.Net] |} Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 560d5d46dadf122fd1c74ca62e8b711a44f829db 2550 2505 2014-01-05T15:09:22Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == {| align="right" {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://alpha.app.net/neilnjae @neilnjae] on App.Net |} Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 1753205daff52ccc248cb270238651a5cefa9ccc 0 1526 2506 2500 2013-12-08T20:34:51Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] fd1ce877a8b38ab333e21313966f62dc2d5ddcf4 2507 2506 2013-12-08T20:45:22Z Neil 2 Added sections for Ruby, Python wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] f62a7dbd1e10cbc1d3c67521f01b43f20cfe1147 2508 2507 2013-12-17T12:19:13Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 88881f59d3323d77a331e64570e27afcf52f6b06 2509 2508 2013-12-30T18:21:39Z Neil 2 /* Additional repositories */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 5cf374edc296a345916999f924fb31f5fc9a7942 2510 2509 2013-12-30T18:26:18Z Neil 2 /* Additional repositories */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] c9e2e065fbf04267b3c4d120777c7c73d54b57ca 2511 2510 2013-12-31T22:43:12Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * VUE * VMWare server * NetLogo * MS core fonts * NetBeans 6.5 (keep the NetBeans 6.1 installation from the Ubuntu package system as that drags a lot of Java bits with it.) * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 2e8927deeaaf790490171c065bb0ebcfe649442f 2512 2511 2014-01-01T11:20:40Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * NetLogo 3.1.5 * JavaNNS VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 5c9e0a7e2c75df5cb3bd18af8d0558ab8debada2 2513 2512 2014-01-01T11:26:41Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == * Add the Partner repositories as sources in the package manager. root@server:~# add-apt-repository “deb http://archive.canonical.com/ lucid partner * Install the Sun Java using the apt package system. root@server:~# apt-get install sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts :You'll need to agree to the licence terms during the installation. * Make it the default with root@server:~# update-java-alternatives -s java-6-sun * Restart Firefox and check the right Java is the default by typing root@desktop:~# java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Server VM (build 16.3-b01, mixed mode) :and, in Firefox, navigating to <code>about:plugins</code> . == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] d347cee98cfe3764ba6401ec9138d2864cf0bf98 2514 2513 2014-01-01T15:05:31Z Neil 2 /* Install and use Sun Java */ Now using WebUpd8 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * sun-java6-jdk sun-java6-jre sun-java6-plugin sun-java6-bin sun-java6-javadb sun-java6-fonts * haskell-platform (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 6b701a8c03102669aa1edfdf386e8ab3d9ddcb81 2515 2514 2014-01-01T16:06:34Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == See [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 706e3f448596e1fa9f7df0ffa54c8e79408025bb 2516 2515 2014-01-01T16:50:47Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Reboot to all the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 See also the [[Ruby on Rails installation]]. ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 5634b9417f995fec1f6a5c80155acbbecf12304c 2517 2516 2014-01-01T16:59:26Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Reboot to all the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 0eaaf159bfcef788b60df0038e20c668e8d5c712 2518 2517 2014-01-01T17:22:53Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Reboot to all the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 7a71505c57077b702cb7ad04ec5daa16d7f63d87 2519 2518 2014-01-01T19:46:04Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Reboot to all the new group permissions to take effect. If you still have problems with using RVM after a reboot, use user@desktop:~$ rvmsudo rvm get stable --auto-dotfiles * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 7700e22c85bf93d1ced4c99b25c8251cf2e51a68 2520 2519 2014-01-01T20:34:24Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts the recommendation form RVM, but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 2a979a9a466dc079d45a0b713d6a366810cd6712 2521 2520 2014-01-01T20:35:13Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * synaptic It gives finer control than Adept (the defualt KDE package manager). Then, using synaptic, install * ubuntu-restricted-extras * kubuntu-restricted-extras :(for MP3 and other playback: it should install a number of other things for you). Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 8f0e2c6f7d6e7e70a929d7e9be6c63a13cc97214 2525 2521 2014-01-02T11:12:08Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * ubuntu-restricted-extras * kubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) Then install these. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 01cfd2258900be3d952589a17912c6b3c97dd4cd 2526 2525 2014-01-02T11:27:56Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * ubuntu-restricted-extras * kubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~$ aptitude install libdvdread4 root@desktop:~$ /usr/share/doc/libdvdread4/install-css.sh Then install these packages. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate<code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] c88cb49160c1eea498897b90ddfdf2e2ee8b96bb 2527 2526 2014-01-02T12:40:25Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === First, install * ubuntu-restricted-extras * kubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~$ aptitude install libdvdread4 root@desktop:~$ /usr/share/doc/libdvdread4/install-css.sh Then install these packages. Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 4b4ea3f57b181c946b0bffc9611c71f37d649b26 2528 2527 2014-01-02T17:36:30Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. * emacs * build-essential * freemind * graphviz * firefox * curl * kdeutils * edubuntu-desktop-kde * edubuntu-addon-kde edubuntu-addon-young edubuntu-addon-science education-logic-games * gnome-games kdegames gnubg * gimp gimp-data-extras gimp-python gimp-plugin-registry * imagemagick * graphicsmagick * libjpeg-progs * gocr-gtk * kmplayer * mplayer mplayer-fonts mencoder * xine-ui xine-plugin * lame * kdemultimedia (?) * kid3 * xinetd * kicker-applets * haskell-platform haskell-platform-prof (supercedes ghc6 (Haskell), ghc6-prof, haskell-mode) * python-all * python3-all * perl * netbeans * swi-prolog-xpce * sbcl (Common Lisp) * slime * kile * kbibtex texlive-latex-base texlive-latex-extra texlive-science * jabref * mmv * kdesvn * git-core git-docs git-svn git-email git-gui gitk * tofrodos * ttf-mscorefonts-installer * curl * virtualbox virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11 === Additional repositories === Include the [https://help.ubuntu.com/community/Medibuntu Medibuntu repositories] to install Win32 codecs and Adobe Acrobat Reader. root@desktop:~# wget http://www.medibuntu.org/sources.list.d/hardy.list \ --output-document=/etc/apt/sources.list.d/medibuntu.list root@desktop:~# apt-get update && apt-get install medibuntu-keyring && apt-get update Then install Acrobat Reader and Skype with the following packages: * acroread * acroread-plugins * mozilla-acroread * skype * w32codecs * libdvdread4 * acidrip * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository Note that after installing <code>libdvdread4</code>, you need to manually finish the install with the command user@desktop:~$ sudo /usr/share/doc/libdvdread4/install-css.sh === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] db2f61181a26e74c38a9f3b5ee7cfe58c95afbb6 2532 2528 2014-01-02T17:51:43Z Neil 2 /* Install additional packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * MongoDB from the extra repository VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 920c38cd77f52e2dbce3e55b8331d303f29a0c0f 2533 2532 2014-01-02T18:14:30Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 480742c1af44e5e5376949d67b19a833a950a095 2534 2533 2014-01-03T11:34:29Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Command aliases == I like having aliases for a few frequently-used Bash commands. Put these aliases at the end of <tt>/etc/bash.bashrc</tt> to ensure that they're available for all users: # some more ls aliases alias ll='ls -l' alias la='ls -la' alias lh='ls -lh' alias lah='ls -lahp' # alias l='ls -CF' alias lsp='ls -p' == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 16f5b5312b3c3a9c753cbe4d5a2db41cb5986f83 2537 2534 2014-01-03T15:05:57Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> == Install VMWare Server == * Download and unpack the .tar.gz file from the [http://www.vmware.com/products/server/ VMWare server page] * Create a user to run the VMWare server: root@desktop:~# adduser --system --group vmware * Give that user a password, so you can log in: root@desktop:~# passwd -u vmware root@desktop:~# passwd vmware * Create a directory to keep the virtual machines: root@desktop:~# mkdir /opt/vmware root@desktop:~# chown vmware:vmware /opt/vmware * Run the installer: root@desktop:~# ./vmware-install.pl * During the install, change the VMWare admin user to <code>vmware</code> * Keep virtual machine files in <code>/opt/vmware/virtual-machines</code> * When installing a virtual machine, have the network adaptor as 'Bridged' More information is on the [https://help.ubuntu.com/community/VMware/Server Ubuntu VMWare Server documentation page]. == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 4cf09c252991938c791bdcc06d624d58dbc84b27 2539 2537 2014-01-03T17:24:35Z Neil 2 /* Install VMWare Server */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt == Enable the Compose key and set AltGr key as Mode_swich == First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] aca04bc6fd1e41f469e1ce657252b8f0d0c8311d 2540 2539 2014-01-03T17:26:06Z Neil 2 /* Enable the Compose key and set AltGr key as Mode_swich */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 655bc8834c3fb74f0e7db142a99b21d976e880b7 2544 2540 2014-01-03T18:05:37Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] 85280a9c505b6d9f61f7759243366debbda34314 2545 2544 2014-01-03T18:09:47Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] ff20f1f996ac4614a72fce2bb2b1cc6622bd5168 2546 2545 2014-01-03T18:10:44Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] c22997fd1ab021428e6ae8d28a6e27e4aa1abb6d 0 1544 2522 2424 2014-01-01T22:57:07Z Neil 2 /* Chrooting the nameservers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <tt>/etc/bind</tt> * Disable rndc by adding this line to the top of <code>/etc/bind/named.conf</code> controls { }; :I could never get it working, and always use the <code>/etc/init.d/bind9</code> init script to control bind. * Alter the <tt>/etc/bind/named.conf.options</tt> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <tt>/etc/bind/named.conf.local</tt> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <tt>/var/cache/bind</tt> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <tt>/etc/bind/</tt>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <tt>/etc/bind/db.domain.tld</tt> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <tt>/etc/bind/db.1.168.192</tt> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update /etc/resolv.conf to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 192.168.2.251 nameserver 192.168.2.252 :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. And don't worry about getting <tt>rndc</tt> working: forcing restarts of BIND seems to work perfectly well. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] d2e59ee62b5a3b54ac97cb5b9077fb39cabe4e5a 2523 2522 2014-01-01T23:02:27Z Neil 2 /* Getting the servers running */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <code>/etc/bind</code> * Alter the <code>/etc/bind/named.conf.options</code> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <code>/etc/bind/named.conf.local</code> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <code>/var/cache/bind</code> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <code>/etc/bind/</code>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <code>/etc/bind/db.domain.tld</code> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <code>/etc/bind/db.1.168.192</code> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update <code>/etc/resolv.conf</code> to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 192.168.2.251 nameserver 192.168.2.252 :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 899b7f2c450f9bde451df5f76356ce92907cb671 0 1515 2524 2490 2014-01-01T23:05:31Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 12.04 LTS and 13.10 (adapted from 8.04 LTS by way of 10.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] b0efe23ac040027b073c4d0e3d3bb55150cca64e 0 1630 2529 2014-01-02T17:39:55Z Neil 2 Created page with "A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages]]. [[Category:Server setup]] acidrip arduino arduino-mk at …" wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity biblatex blueman calibre cheese cmake couchdb-bin create-resources cython dos2unix dpkg-dev easytag eog fbreader ffmpeg fgetty finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing fop gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gmusicbrowser gnome-games gnubg gnuplot-qt gocr gparted graphicsmagick gtk2-engines-qtcurve gucharmap guvcview hal-info haskell-mode haskell-platform haskell-platform-prof hplip-dbg hplip-gui html2text ibus idle idle3 inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs normalize-audio octave ogmtools openshot p7zip pandoc paprefs pastebinit pavucontrol pavumeter pdftk perlmagick phonon phonon-backend-gstreamer pitivi postgresql postgresql-client pychess python-all python3-all r-base r-recommended samba sbcl scratch screen scribus scribus-doc scribus-template seahorse shotwell slime sni-qt software-center software-properties-gtk sound-juicer sqlite3 stellarium tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science thunderbird tk-dev tk tofrodos traceroute tuxpaint tuxpaint-config units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xsane xtightvncviewer zenity 40836d8b2d7e3a77c6ee4f292c59032f5052f854 2530 2529 2014-01-02T17:42:05Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity biblatex blueman calibre cheese cmake couchdb-bin create-resources cython dos2unix dpkg-dev easytag eog fbreader ffmpeg fgetty finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing fop gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gmusicbrowser gnome-games gnubg gnuplot-qt gocr gparted graphicsmagick gtk2-engines-qtcurve gucharmap guvcview hal-info haskell-mode haskell-platform haskell-platform-prof hplip-dbg hplip-gui html2text ibus idle idle3 inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs normalize-audio octave ogmtools openshot p7zip pandoc paprefs pastebinit pavucontrol pavumeter pdftk perlmagick phonon phonon-backend-gstreamer pitivi postgresql postgresql-client pychess python-all python3-all r-base r-recommended samba sbcl scratch screen scribus scribus-doc scribus-template seahorse shotwell slime sni-qt software-center software-properties-gtk sound-juicer sqlite3 stellarium tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science thunderbird tk-dev tk tofrodos traceroute tuxpaint tuxpaint-config units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xsane xtightvncviewer zenity 944b1eaf54d4c40f269a34a33666b56a923fcbaf 2531 2530 2014-01-02T17:49:37Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity biblatex blueman build-essential calibre cheese cmake couchdb-bin create-resources cython dos2unix dpkg-dev easytag eog fbreader ffmpeg fgetty finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing fop gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gmusicbrowser gnome-games gnubg gnuplot-qt gocr gparted graphicsmagick gtk2-engines-qtcurve gucharmap guvcview hal-info haskell-mode haskell-platform haskell-platform-prof hplip-dbg hplip-gui html2text ibus idle idle3 inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs normalize-audio octave ogmtools openshot p7zip pandoc paprefs pastebinit pavucontrol pavumeter pdftk perlmagick phonon phonon-backend-gstreamer pitivi postgresql postgresql-client pychess python-all python3-all r-base r-recommended samba sbcl scratch screen scribus scribus-doc scribus-template seahorse shotwell slime sni-qt software-center software-properties-gtk sound-juicer sqlite3 stellarium tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science thunderbird tk-dev tk tofrodos traceroute tuxpaint tuxpaint-config units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xsane xtightvncviewer zenity f11fb2fd08192ef5588034e08364c5b44c1db71f 2542 2531 2014-01-03T17:42:13Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity biblatex blueman build-essential calibre cheese cmake couchdb-bin create-resources cython dos2unix dpkg-dev easytag eog fbreader ffmpeg fgetty finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing fop gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gmusicbrowser gnome-games gnubg gnuplot-qt gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve gucharmap guvcview hal-info haskell-mode haskell-platform haskell-platform-prof hplip-dbg hplip-gui html2text ibus idle idle3 inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs normalize-audio octave ogmtools openshot p7zip pandoc paprefs pastebinit pavucontrol pavumeter pdftk perlmagick phonon phonon-backend-gstreamer pitivi postgresql postgresql-client pychess python-all python3-all r-base r-recommended samba sbcl scratch screen scribus scribus-doc scribus-template seahorse shotwell slime sni-qt software-center software-properties-gtk sound-juicer sqlite3 stellarium tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science thunderbird tk-dev tk tofrodos traceroute tuxpaint tuxpaint-config units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xsane xtightvncviewer zenity 958c6e8355607b1e060cc4025f9be7557fec54fc 2543 2542 2014-01-03T17:43:48Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity biblatex blueman build-essential calibre cheese cmake couchdb-bin create-resources cython dos2unix dpkg-dev easytag eog fbreader ffmpeg fgetty finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing fop gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gmusicbrowser gnome-games gnubg gnuplot-qt gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve gucharmap guvcview hal-info haskell-mode haskell-platform haskell-platform-prof hplip-dbg hplip-gui html2text ibus idle idle3 inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs normalize-audio octave ogmtools openshot p7zip pandoc paprefs pastebinit pavucontrol pavumeter pdftk perlmagick phonon phonon-backend-gstreamer pitivi postgresql postgresql-client pychess python-all python3-all r-base r-recommended samba sbcl scratch screen scribus scribus-doc scribus-template seahorse shotwell slime sni-qt software-center software-properties-gtk sound-juicer sqlite3 stellarium swi-prolog tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science thunderbird tk-dev tk tofrodos traceroute tuxpaint tuxpaint-config units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xsane xtightvncviewer zenity b41e9ffdfc986e1e40351523663f1c1aac57a647 0 1577 2535 2375 2014-01-03T12:44:04Z Neil 2 /* Installation on Ubuntu Linux */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. Install [[Miscellaneous setup#Install_Ruby|Ruby using RVM]]. * Install the following packages: :Server root@server:~# apt-get install build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby \ libmysqlclient-dev \ sqlite3 libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev :Desktop root@desktop:~# apt-get install build-essential libmysqlclient-dev \ sqlite3 libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev ==RVM== If using RVM to keep different Ruby versions around, you need to install the Readline development headers first: root@desktop:~# apt-get install libreadline-dev user@desktop:~$ gem install rails rake rcov rspec gemcutter git jeweler capistrano mysql sqlite3 shoulda cucumber yard :If any of the packages fail to install, simply re-install them and they should work fine. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 == Installing Rails with NetBeans 6.5 on Windows == * Download and install Netbeans 6.5 * Set the JAVA_HOME environment variable ** Right click the 'My Computer' icon on the desktop. Select 'Properties' ** Select the 'Advanced' tab, press the 'Environment variables' button ** Add a new System variable, <code>JAVA_HOME</code>, which contains the path of the NetBeans JDK (currently <code>C:\Program Files\Java\jdk1.6.0_03</code>: use NetBeans's Tools | Java Platform menu item to find the path) ::(See [http://wso2.org/project/wsas/java/1.1/docs/setting-java-home.html here] for pretty pictures) * Update RubyGems ** Open a new command terminal (Start | Run | cmd) ** CD to C:\Program Files\NetBeans 6.5\ruby2\jruby-1.1.4\bin\ ** Update RubyGems gem install rubygems-update rubygems-update gem update --system gem --version ** Check you're at version 1.3.5 * Update Rails. Stay in the same directory and give the command gem update ** Check Rails is at version 2.3.3 * Install the Ruby MySQL gem gem install mysql == Getting NetBeans 6.5 to use another Ruby == We'll be using the system-wide Ruby and Gem programs, rather than NetBeans's built-in JRuby ones. To make NetBeans know where they are, use the Tools | Ruby Platforms menu item and tell NetBeans about the Ruby executable in /usr/bin/ruby and /usr/bin/gem. Do the same for Tools | Ruby gems. Then tell NetBeans which Ruby platform to use. You need to do this on a project-by-project basis. Right-click on the project's root node in the NetBeans navigator, select the Set Configuration | Customize... option and select the right Ruby platform from the drop-down list at the top of the dialog. Note that WEBrick runs NetBeans's 'main project' when a project is run. When creating a new project, the database needs to be created. Either do this from the Instant Rails Ruby console, or use the NetBeans MySQL database connectors. (Does this latter approach work?) [[Category:Ruby]] [[Category:Rails]] a460e0e4146b59436d897ad47e0ead38feba62a1 2536 2535 2014-01-03T12:45:45Z Neil 2 wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. Install [[Miscellaneous setup#Install_Ruby|Ruby using RVM]]. * Install the following packages: :Server root@server:~# apt-get install build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby \ libmysqlclient-dev \ sqlite3 libsqlite3-dev imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev :Desktop root@desktop:~# apt-get install build-essential libmysqlclient-dev \ sqlite3 libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev ==Passenger== * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 ==See also== * [http://gorails.com/setup/ubuntu/13.10 Setup Ruby On Rails on Ubuntu 13.10] [[Category:Ruby]] [[Category:Rails]] 3181256771d5e2637f649b3845c780ebd46667df 2538 2536 2014-01-03T15:20:30Z Neil 2 Added postres packages wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. Install [[Miscellaneous setup#Install_Ruby|Ruby using RVM]]. * Install the following packages: :Server root@server:~# apt-get install build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby \ libmysqlclient-dev \ postgresql postgresql-client libpq-dev \ sqlite3 libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev :Desktop root@desktop:~# apt-get install build-essential libmysqlclient-dev \ sqlite3 libsqlite3-dev \ postgresql postgresql-client libpq-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev ==Passenger== * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 ==See also== * [http://gorails.com/setup/ubuntu/13.10 Setup Ruby On Rails on Ubuntu 13.10] [[Category:Ruby]] [[Category:Rails]] 89c9ab5c8aa2ca165f29c7781f6a43bf6ea48b7d 2541 2538 2014-01-03T17:38:34Z Neil 2 /* Passenger */ wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. Install [[Miscellaneous setup#Install_Ruby|Ruby using RVM]]. * Install the following packages: :Server root@server:~# apt-get install build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby \ libmysqlclient-dev \ postgresql postgresql-client libpq-dev \ sqlite3 libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev :Desktop root@desktop:~# apt-get install build-essential libmysqlclient-dev \ sqlite3 libsqlite3-dev \ postgresql postgresql-client libpq-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev ==Passenger== Follow the [http://www.modrails.com/documentation/Users%20guide%20Apache.html#install_on_debian_ubuntu instructions from Phusion]. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 ==See also== * [http://gorails.com/setup/ubuntu/13.10 Setup Ruby On Rails on Ubuntu 13.10] [[Category:Ruby]] [[Category:Rails]] 52fa1d2627ae3242f3b29e798a68936df643b13b 0 1578 2547 2125 2014-01-04T11:47:00Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postresql * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role feedcatcher with createdb login password 'feedcatcher' Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] a3a52f59252fbef85e3ebaa86ecb9888317afb46 2548 2547 2014-01-04T11:48:31Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postresql * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] 3f1c0a9b52cf1972311979a152640d7140653f3e 2549 2548 2014-01-04T14:31:53Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postresql * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] 9dbceaea11fa140236f6d7c1ac34bb2d0f0789d8 2553 2549 2014-01-06T08:37:44Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postresql * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end gem 'capistrano', group: :development group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install RSpec-Rails support: user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ bundle exec rspec Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] 0da5805be0ddbfabad884a2685f7bbe22d134148 0 1631 2551 2014-01-05T18:59:24Z Neil 2 Created page with "{|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]'…" wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client pgadmin3 == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following line: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD host all all 192.168.1.0 255.255.255.0 md5 Edit <code>/etc/postgresql/8.3/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] 35735ffb39e9b99a09a2cbe7a8b1c6f78794a25d 2552 2551 2014-01-05T19:10:04Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client pgadmin3 == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/8.3/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] b525b04d807b95abf1edd3dd67e80280f0809d60 0 1578 2554 2553 2014-01-06T08:40:51Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end gem 'capistrano', group: :development group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install RSpec-Rails support: user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ bundle exec rspec Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] c4ee2805f0d1c2651acbb78e5b842b76021d726c 2555 2554 2014-01-06T19:31:45Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end gem 'capistrano', group: :development group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install RSpec-Rails support: user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. [[Category:Ruby]] [[Category:Rails]] 32adc4e3293b0d79482a2080782e1d94c6914831 2556 2555 2014-01-06T19:36:03Z Neil 2 /* See also */ wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end gem 'capistrano', group: :development group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install RSpec-Rails support: user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. * [http://edgeguides.rubyonrails.org/index.html Rails guides] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres Setting up Ruby on Rails with Postgres] * [https://github.com/rspec/rspec-rails Rspec-Rails] * [http://guides.rubyonrails.org/testing.html Rails testing] * [http://betterspecs.org/ BetterSpecs] (RSpec style guide) [[Category:Ruby]] [[Category:Rails]] 79dfbf7749306d1c67b62492f7dbd04473244540 2559 2556 2014-01-06T20:34:15Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project * Import the <code>.gitignore</code>: For Rails, use this one from Github, below. Then initialise the repo: user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postgresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end gem 'capistrano', group: :development group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install RSpec-Rails support: user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec ==Github's .gitignore file== *.rbc *.sassc .sass-cache capybara-*.html .rspec /log /tmp /db/*.sqlite3 /db/*.sqlite3-journal /public/system /coverage/ /spec/tmp **.orig rerun.txt pickle-email-*.html config/initializers/secret_token.rb config/secrets.yml ## Environment normalisation: /.bundle /vendor/bundle # these should all be checked in to normalise the environment: # Gemfile.lock, .ruby-version, .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc ## Github's global gitignore for Ruby *.gem *.rbc /.config /coverage/ /InstalledFiles /pkg/ /spec/reports/ /test/tmp/ /test/version_tmp/ /tmp/ ## Documentation cache and generated files: /.yardoc/ /_yardoc/ /doc/ /rdoc/ ## Environment normalisation: /.bundle/ /lib/bundler/man/ # for a library or gem, you might want to ignore these files since the code is # intended to run in multiple environments; otherwise, check them in: # Gemfile.lock # .ruby-version # .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. * [http://edgeguides.rubyonrails.org/index.html Rails guides] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres Setting up Ruby on Rails with Postgres] * [https://github.com/rspec/rspec-rails Rspec-Rails] * [http://guides.rubyonrails.org/testing.html Rails testing] * [http://betterspecs.org/ BetterSpecs] (RSpec style guide) [[Category:Ruby]] [[Category:Rails]] b684b7b09c2b3ab1f1180cdc02c4daa9861af5d3 2563 2559 2014-01-06T22:32:15Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project * Import the <code>.gitignore</code>: For Rails, use this one from Github, below. Then initialise the repo: user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postgresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end gem 'capistrano', group: :development group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install RSpec-Rails support: user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * Create and migrate the database user@desktop:~/project$ rake db:setup user@desktop:~/project$ rake db:migrate * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec ==Github's .gitignore file== *.rbc *.sassc .sass-cache capybara-*.html .rspec /log /tmp /db/*.sqlite3 /db/*.sqlite3-journal /public/system /coverage/ /spec/tmp **.orig rerun.txt pickle-email-*.html config/initializers/secret_token.rb config/secrets.yml ## Environment normalisation: /.bundle /vendor/bundle # these should all be checked in to normalise the environment: # Gemfile.lock, .ruby-version, .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc ## Github's global gitignore for Ruby *.gem *.rbc /.config /coverage/ /InstalledFiles /pkg/ /spec/reports/ /test/tmp/ /test/version_tmp/ /tmp/ ## Documentation cache and generated files: /.yardoc/ /_yardoc/ /doc/ /rdoc/ ## Environment normalisation: /.bundle/ /lib/bundler/man/ # for a library or gem, you might want to ignore these files since the code is # intended to run in multiple environments; otherwise, check them in: # Gemfile.lock # .ruby-version # .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. * [http://edgeguides.rubyonrails.org/index.html Rails guides] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres Setting up Ruby on Rails with Postgres] * [https://github.com/rspec/rspec-rails Rspec-Rails] * [http://guides.rubyonrails.org/testing.html Rails testing] * [http://betterspecs.org/ BetterSpecs] (RSpec style guide) [[Category:Ruby]] [[Category:Rails]] 072466a712ad3c5c1b65ee15142d6bf6b77650a6 2569 2563 2014-01-08T20:13:29Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project * Import the <code>.gitignore</code>: For Rails, use this one from Github, below. Then initialise the repo: user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postgresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end group :development do gem 'capistrano', '~> 3.0' gem 'capistrano-rails', '~> 1.1' gem 'capistrano-bundler' gem 'capistrano-rvm' end group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install RSpec-Rails support: user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * Create and migrate the database user@desktop:~/project$ rake db:setup user@desktop:~/project$ rake db:migrate * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec ==Github's .gitignore file== *.rbc *.sassc .sass-cache capybara-*.html .rspec /log /tmp /db/*.sqlite3 /db/*.sqlite3-journal /public/system /coverage/ /spec/tmp **.orig rerun.txt pickle-email-*.html config/initializers/secret_token.rb config/secrets.yml ## Environment normalisation: /.bundle /vendor/bundle # these should all be checked in to normalise the environment: # Gemfile.lock, .ruby-version, .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc ## Github's global gitignore for Ruby *.gem *.rbc /.config /coverage/ /InstalledFiles /pkg/ /spec/reports/ /test/tmp/ /test/version_tmp/ /tmp/ ## Documentation cache and generated files: /.yardoc/ /_yardoc/ /doc/ /rdoc/ ## Environment normalisation: /.bundle/ /lib/bundler/man/ # for a library or gem, you might want to ignore these files since the code is # intended to run in multiple environments; otherwise, check them in: # Gemfile.lock # .ruby-version # .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. * [http://edgeguides.rubyonrails.org/index.html Rails guides] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres Setting up Ruby on Rails with Postgres] * [https://github.com/rspec/rspec-rails Rspec-Rails] * [http://guides.rubyonrails.org/testing.html Rails testing] * [http://betterspecs.org/ BetterSpecs] (RSpec style guide) [[Category:Ruby]] [[Category:Rails]] 8f596030c2508a2a34b1f8a51d004854f92cb92d 2570 2569 2014-01-08T20:14:38Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project * Import the <code>.gitignore</code>: For Rails, use this one from Github, below. Then initialise the repo: user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postgresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end group :development do gem 'capistrano', '~> 3.0' gem 'capistrano-rails', '~> 1.1' gem 'capistrano-bundler' gem 'capistrano-rvm' end group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install the bundle and RSpec-Rails support: user@desktop:~/project$ bundle install user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * Create and migrate the database user@desktop:~/project$ rake db:setup user@desktop:~/project$ rake db:migrate * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec ==Github's .gitignore file== *.rbc *.sassc .sass-cache capybara-*.html .rspec /log /tmp /db/*.sqlite3 /db/*.sqlite3-journal /public/system /coverage/ /spec/tmp **.orig rerun.txt pickle-email-*.html config/initializers/secret_token.rb config/secrets.yml ## Environment normalisation: /.bundle /vendor/bundle # these should all be checked in to normalise the environment: # Gemfile.lock, .ruby-version, .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc ## Github's global gitignore for Ruby *.gem *.rbc /.config /coverage/ /InstalledFiles /pkg/ /spec/reports/ /test/tmp/ /test/version_tmp/ /tmp/ ## Documentation cache and generated files: /.yardoc/ /_yardoc/ /doc/ /rdoc/ ## Environment normalisation: /.bundle/ /lib/bundler/man/ # for a library or gem, you might want to ignore these files since the code is # intended to run in multiple environments; otherwise, check them in: # Gemfile.lock # .ruby-version # .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. * [http://edgeguides.rubyonrails.org/index.html Rails guides] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres Setting up Ruby on Rails with Postgres] * [https://github.com/rspec/rspec-rails Rspec-Rails] * [http://guides.rubyonrails.org/testing.html Rails testing] * [http://betterspecs.org/ BetterSpecs] (RSpec style guide) [[Category:Ruby]] [[Category:Rails]] 837f3cf9511c63908fa0101e96572d0ac2f4b50c 2571 2570 2014-01-08T20:16:55Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project * Import the <code>.gitignore</code>: For Rails, use this one from Github, below. Then initialise the repo: user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postgresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end group :development do gem 'capistrano', '~> 3.0' gem 'capistrano-rails', '~> 1.1' gem 'capistrano-bundler','>= 1.1.0' gem 'capistrano-rvm' end group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install the bundle and RSpec-Rails support: user@desktop:~/project$ bundle install user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword' * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * Create and migrate the database user@desktop:~/project$ rake db:setup user@desktop:~/project$ rake db:migrate * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec ==Github's .gitignore file== *.rbc *.sassc .sass-cache capybara-*.html .rspec /log /tmp /db/*.sqlite3 /db/*.sqlite3-journal /public/system /coverage/ /spec/tmp **.orig rerun.txt pickle-email-*.html config/initializers/secret_token.rb config/secrets.yml ## Environment normalisation: /.bundle /vendor/bundle # these should all be checked in to normalise the environment: # Gemfile.lock, .ruby-version, .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc ## Github's global gitignore for Ruby *.gem *.rbc /.config /coverage/ /InstalledFiles /pkg/ /spec/reports/ /test/tmp/ /test/version_tmp/ /tmp/ ## Documentation cache and generated files: /.yardoc/ /_yardoc/ /doc/ /rdoc/ ## Environment normalisation: /.bundle/ /lib/bundler/man/ # for a library or gem, you might want to ignore these files since the code is # intended to run in multiple environments; otherwise, check them in: # Gemfile.lock # .ruby-version # .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. * [http://edgeguides.rubyonrails.org/index.html Rails guides] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres Setting up Ruby on Rails with Postgres] * [https://github.com/rspec/rspec-rails Rspec-Rails] * [http://guides.rubyonrails.org/testing.html Rails testing] * [http://betterspecs.org/ BetterSpecs] (RSpec style guide) [[Category:Ruby]] [[Category:Rails]] 26b874531d27a980cc38f5130b9d7aa7dc6560ce 2579 2571 2014-01-10T20:22:21Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project * Import the <code>.gitignore</code>: For Rails, use this one from Github, below. Then initialise the repo: user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postgresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end group :development do gem 'capistrano', '~> 3.0' gem 'capistrano-rails', '~> 1.1' gem 'capistrano-bundler','>= 1.1.0' gem 'capistrano-rvm' end group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install the bundle and RSpec-Rails support: user@desktop:~/project$ bundle install user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword'; * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * Create and migrate the database user@desktop:~/project$ rake db:setup user@desktop:~/project$ rake db:migrate * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec ==Github's .gitignore file== *.rbc *.sassc .sass-cache capybara-*.html .rspec /log /tmp /db/*.sqlite3 /db/*.sqlite3-journal /public/system /coverage/ /spec/tmp **.orig rerun.txt pickle-email-*.html config/initializers/secret_token.rb config/secrets.yml ## Environment normalisation: /.bundle /vendor/bundle # these should all be checked in to normalise the environment: # Gemfile.lock, .ruby-version, .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc ## Github's global gitignore for Ruby *.gem *.rbc /.config /coverage/ /InstalledFiles /pkg/ /spec/reports/ /test/tmp/ /test/version_tmp/ /tmp/ ## Documentation cache and generated files: /.yardoc/ /_yardoc/ /doc/ /rdoc/ ## Environment normalisation: /.bundle/ /lib/bundler/man/ # for a library or gem, you might want to ignore these files since the code is # intended to run in multiple environments; otherwise, check them in: # Gemfile.lock # .ruby-version # .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc Some steps to go through when creating a new Ruby on Rails project. == Subversion control setup == Assume you're using NetBeans IDE for development. * Some files should be excluded from version control. Add the <code>svn:ignore</code> to the following directories, with the given value. {| align="center" {{Prettytable}} |- |{{Hl2}} |'''Directory''' |{{Hl2}} |'''Value''' |- | log || *.log |- | config || database.yml |- | tmp || * |- | doc || *doc |} :then copy <code>config/database.yml</code> to <code>config/database.sample.yml</code>, which ''will'' be included in the repository. :If you're doing this on Netbeans, you may need to remove these files from Netbeans's first commit queue. The idea is that each developer (if you're collaborating) will use their own <code>database.yml</code> file. It also means that the database password for your production server isn't exposed in the Subversion repository. * Mark some files as being executable. NetBeans should do this for you on the first commit. If not, add the <code>svn:executable</code> property (with value '*') to every file in the <code>scripts</code> directory and to all the <code>public/dispatch.*</code> files. == Database creation == * Log into the MySQL server user@desktop:~# mysql -u root -p === Development and test databases === For the development project, where the user is restricted to the localhost only * Create the database mysql> create database project_development; mysql> create user 'project'@'localhost'; mysql> grant all on project_development.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_development port: 3306 username: project password: socket: /var/run/mysqld/mysqld.sock :Rails assumes the user is on the localhost anyway. :On Red Hat/Fedora boxes, the MySQL socket may be at <code>/var/lib/mysql/mysql.sock</code> instead. * If you want to use Sqlite3, use this stanza instead: test: adapter: sqlite3 encoding: utf8 reconnect: false database: db/test.sqlite3 pool: 5 timeout: 5000 === Production database === For the production project, where the user can log in from any host but access is protected by a password * Create the database mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'password'; mysql> grant all on project_production.* to 'project'@'localhost'; * In <code>database.yaml</code>, create the relevant details: development: host: localhost adapter: mysql database: project_production port: 3306 username: project password: password == Enable log rotation for production servers == Modify <code>config/environments/production.rb</code> to include these lines: # Rotate logs when the file grows to 10Mb, keep 10 old log files config.logger = Logger.new(config.log_path, 10, 10.megabytes) == Change the welcome page == * Pick the controller you want to handle requests at the root of your website. For instance, for the demo Depot application, you'll want the <code>store</code> controller to handle HTTP requests of <code><nowiki>http://www.depot.tld/</nowiki></code>, as well as the more usual <code><nowiki>http://www.depot.tld/store/</nowiki></code>. * Uncomment the <code>map.root</code> line in <code>config/routes.rb</code> and insert the name of the controller: map.root :controller => "store" * Remove or welcome page, <code>public/index.html</code> * If you now point a browser at <code><nowiki>http://localhost:3000/</nowiki></code>, you should see the page served by the controller you chose! == See also == There are a couple of frameworks with most of the obvious plugin-installation already done. * [http://jimneath.org/2008/09/09/bort-base-rails-application/ Bort] is a standard Rails app with authentication, pagination, and the like already included. * [http://github.com/insoshi/insoshi/tree/master Insoshi] is a basic social networking application. * [http://edgeguides.rubyonrails.org/index.html Rails guides] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres Setting up Ruby on Rails with Postgres] * [https://github.com/rspec/rspec-rails Rspec-Rails] * [http://guides.rubyonrails.org/testing.html Rails testing] * [http://betterspecs.org/ BetterSpecs] (RSpec style guide) [[Category:Ruby]] [[Category:Rails]] e8ba35c7bcab5ff79c44c6ea063314d17d99f347 2603 2579 2014-01-13T09:50:00Z Neil 2 wikitext text/x-wiki Creating a new Rails 4 project. * Check that the <code>.gitignore_global</code> file contains the files you want ignored. * Create the skeleton of a new Rails app user@desktop:~$ mkdir project user@desktop:~$ cd project * Import the <code>.gitignore</code>: For Rails, use this one from Github, below. Then initialise the repo: user@desktop:~/project$ git init user@desktop:~/project$ git add . user@desktop:~/project$ rails new . --git --database=postgresql --skip-test-unit * Modify the <code>Gemspec</code> file to refer to RSpec in the test evironment: gem 'rails', '4.0.2' gem 'pg' gem 'sass-rails', '~> 4.0.0' gem 'uglifier', '>= 1.3.0' gem 'coffee-rails', '~> 4.0.0' gem 'jquery-rails' gem 'turbolinks' gem 'jbuilder', '~> 1.2' group :doc do gem 'sdoc', require: false end group :development do gem 'capistrano', '~> 3.0' gem 'capistrano-rails', '~> 1.1' gem 'capistrano-bundler','>= 1.1.0' gem 'capistrano-rvm' end group :development, :test do gem "rspec-rails" end group :test do gem "factory_girl_rails" gem "cucumber-rails", :require => false gem "capybara" gem "database_cleaner" gem "launchy" end :then install the bundle and RSpec-Rails support: user@desktop:~/project$ bundle install user@desktop:~/project$ rails generate rspec:install * Create the database user for this project (development and test on the local machine) user@desktop:~$ sudo -u postgres psql postgres=# create role projectdbuser with createdb login password 'projectdbpassword'; * Edit <code>config/database.yml</code> to include the passwords for the development and test databases. Add <code>host: localhost</code> lines for the development and test databases. * Create and migrate the database user@desktop:~/project$ rake db:setup user@desktop:~/project$ rake db:migrate * When running tests, migrate the test database (if necessary) and then run the tests: user@desktop:~/project$ rake db:migrate RAILS_ENV=test user@desktop:~/project$ rake spec ==Github's .gitignore file== *.rbc *.sassc .sass-cache capybara-*.html .rspec /log /tmp /db/*.sqlite3 /db/*.sqlite3-journal /public/system /coverage/ /spec/tmp **.orig rerun.txt pickle-email-*.html config/initializers/secret_token.rb config/secrets.yml ## Environment normalisation: /.bundle /vendor/bundle # these should all be checked in to normalise the environment: # Gemfile.lock, .ruby-version, .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc ## Github's global gitignore for Ruby *.gem *.rbc /.config /coverage/ /InstalledFiles /pkg/ /spec/reports/ /test/tmp/ /test/version_tmp/ /tmp/ ## Documentation cache and generated files: /.yardoc/ /_yardoc/ /doc/ /rdoc/ ## Environment normalisation: /.bundle/ /lib/bundler/man/ # for a library or gem, you might want to ignore these files since the code is # intended to run in multiple environments; otherwise, check them in: # Gemfile.lock # .ruby-version # .ruby-gemset # unless supporting rvm < 1.11.0 or doing something fancy, ignore this: .rvmrc ==See also== * [http://edgeguides.rubyonrails.org/index.html Rails guides] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres Setting up Ruby on Rails with Postgres] * [https://github.com/rspec/rspec-rails Rspec-Rails] * [http://guides.rubyonrails.org/testing.html Rails testing] * [http://betterspecs.org/ BetterSpecs] (RSpec style guide) [[Category:Ruby]] [[Category:Rails]] 3d55ae233a4491a02b4d9eb4ecadf306495504fa 0 1577 2557 2541 2014-01-06T19:37:31Z Neil 2 wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. == Installation on Ubuntu Linux == If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. Install [[Miscellaneous setup#Install_Ruby|Ruby using RVM]]. * Install the following packages: :Server root@server:~# apt-get install build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby \ libmysqlclient-dev \ postgresql postgresql-client libpq-dev \ sqlite3 libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev :Desktop root@desktop:~# apt-get install build-essential libmysqlclient-dev \ sqlite3 libsqlite3-dev \ postgresql postgresql-client libpq-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev ==Passenger== Follow the [http://www.modrails.com/documentation/Users%20guide%20Apache.html#install_on_debian_ubuntu instructions from Phusion]. * To install Phusion Passenger, give the command root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module :and follow the instructions. The script will compile and install a new Apache2 module for you. :You'll need to add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 ==See also== * [http://gorails.com/setup/ubuntu/13.10 Setup Ruby On Rails on Ubuntu 13.10] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres How To Setup Ruby on Rails with Postgres] [[Category:Ruby]] [[Category:Rails]] 5a71e9dea87961543575e47068de56d38fee19c9 2601 2557 2014-01-13T09:46:48Z Neil 2 wikitext text/x-wiki How to install Ruby, Gems, and Rails. Instructions for Linux workstation, Linux server, and Windows workstation. * If MySQL isn't already installed, [[MySQL configuration|install and configure it]]. * If MySQL isn't already installed, [[PostgreSQL configuration|install and configure it]]. * Install [[Miscellaneous setup#Install_Ruby|Ruby using RVM]]. == Installation on Ubuntu Linux == * Install the following packages: :Server root@server:~# apt-get install build-essential apache2 apache2-mpm-prefork \ apache2-prefork-dev libapache2-mod-ruby \ libmysqlclient-dev \ postgresql postgresql-client libpq-dev \ sqlite3 libsqlite3-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev :Desktop root@desktop:~# apt-get install build-essential libmysqlclient-dev \ sqlite3 libsqlite3-dev \ postgresql postgresql-client libpq-dev \ imagemagick graphicsmagick libmagick10 libgraphicsmagick1 libgraphicsmagick1-dev ==Passenger== Follow the [http://www.modrails.com/documentation/Users%20guide%20Apache.html#install_on_debian_ubuntu instructions from Phusion]. root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7 root@desktop:~# aptitude install apt-transport-https root@desktop:~# echo "deb https://oss-binaries.phusionpassenger.com/apt/passenger saucy main" >> /etc/apt/sources.list.d/passenger.list root@desktop:~# chown root:root /etc/apt/sources.list.d/passenger.list root@desktop:~# chmod 600 /etc/apt/sources.list.d/passenger.list root@desktop:~# aptitude update root@desktop:~# aptitude install libapache2-mod-passenger ==See also== * [http://gorails.com/setup/ubuntu/13.10 Setup Ruby On Rails on Ubuntu 13.10] * [https://www.digitalocean.com/community/articles/how-to-setup-ruby-on-rails-with-postgres How To Setup Ruby on Rails with Postgres] [[Category:Ruby]] [[Category:Rails]] fab3fc8d30807728bf9f4eb0230153c5572ed900 0 1631 2558 2552 2014-01-06T19:40:03Z Neil 2 /* See also */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client pgadmin3 == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/8.3/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] 12cb403febce00bd7ad319d304ec36767e91ffb7 2560 2558 2014-01-06T21:16:17Z Neil 2 /* Remote access */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client pgadmin3 == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf</code> and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/8.3/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] 5bec365a9605957489604766f02d6a162105ebc2 2561 2560 2014-01-06T21:17:44Z Neil 2 /* Remote access */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client pgadmin3 == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf</code> and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/9.1/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] e5b3a00d7e34b83eb8aedb8ef54e4b3b1272a0c8 0 1579 2562 2193 2014-01-06T22:15:37Z Neil 2 wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] a971a0fbdf4839f74387238265e8785ba7313ffb 2565 2562 2014-01-08T19:45:18Z Neil 2 wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code> deploy user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] c657b79af3b3fa4dc423f4054a0849fcfa6dddc7 2566 2565 2014-01-08T19:53:18Z Neil 2 wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code> deploy user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment ===Capify the project=== ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] cf2cb2c7c6c8517edc474f3d5e81e22cc0726bc8 2567 2566 2014-01-08T19:54:12Z Neil 2 /* One-time setup: create the deploy user */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment ===Capify the project=== ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 4545f4c91c7f0c3916693edfce1f7a15f88504b3 2568 2567 2014-01-08T19:59:17Z Neil 2 /* Per-project setup */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld/project.git' set :branch, 'master' ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 5476ba8b7150682aa144a87ee123606fe8c55c26 2572 2568 2014-01-08T22:50:41Z Neil 2 wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld/project.git' set :branch, 'master' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 918813bf0c4228e5894441ced75ca5dfd7843b25 2573 2572 2014-01-08T23:05:23Z Neil 2 /* Capify the project */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld/project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 941f7ba929a30bb1e2bc00cd8b0d267d16db0640 2574 2573 2014-01-08T23:14:33Z Neil 2 /* Subsequent deployments */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld/project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] a8a949e8efdfda7b64f15a70e6b226eca8f9afb7 2575 2574 2014-01-08T23:14:52Z Neil 2 /* Capify the project */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 48529a2de358c430408d151565a8b421d4398b5a 2576 2575 2014-01-08T23:16:44Z Neil 2 /* Capify the project */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(This omits the default "staging" deploy target) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] a957fd9c7ea9e87ff2676f40e37481a45701abe4 2577 2576 2014-01-08T23:17:13Z Neil 2 /* Capify the project */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 0b7ce3b90a068f7d03ce23c25dc2ac223f87677c 2578 2577 2014-01-08T23:31:10Z Neil 2 /* See also */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 5f9a8583401b7046d155ea1898d50f2abb9adef2 2580 2578 2014-01-10T20:27:09Z Neil 2 /* Create the production database */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role project with createdb login password 'projectpassword'; CREATE ROLE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] cc892956068dd8551c1d781cf801113da83049a5 2581 2580 2014-01-10T20:27:19Z Neil 2 /* Create the production database */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role project with createdb login password 'projectpassword'; CREATE ROLE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] c1405c657602db927ad0278226621cf156db1826 2582 2581 2014-01-10T20:39:27Z Neil 2 /* Create the Apache2 virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld} :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld <Directory /var/www/project.domain.tld/public> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4096 Jan 10 20:35 . drwxr-xr-x 13 root root 4096 Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4096 Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4096 Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role project with createdb login password 'projectpassword'; CREATE ROLE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 1a4dcc645e9e2882a64d2c9d5537d96d2036515a 2583 2582 2014-01-10T20:51:02Z Neil 2 /* Capify the project */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld <Directory /var/www/project.domain.tld/public> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4096 Jan 10 20:35 . drwxr-xr-x 13 root root 4096 Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4096 Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4096 Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role project with createdb login password 'projectpassword'; CREATE ROLE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 9e0a04a7af94b5d9144dbe46e1a7332e6192535a 2584 2583 2014-01-10T20:56:28Z Neil 2 /* Capify the project */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld <Directory /var/www/project.domain.tld/public> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4096 Jan 10 20:35 . drwxr-xr-x 13 root root 4096 Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4096 Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4096 Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role project with createdb login password 'projectpassword'; CREATE ROLE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 7f41a3e6b94125050d3c0cdf57c456c316e15a9a 2585 2584 2014-01-10T21:00:55Z Neil 2 /* Create the Apache2 virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld <Directory /var/www/project.domain.tld/public> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role project with createdb login password 'projectpassword'; CREATE ROLE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 57fc27b86b89fd4619136355b45adc9d67e56bd9 2586 2585 2014-01-10T21:06:57Z Neil 2 /* Make the first deployment */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld <Directory /var/www/project.domain.tld/public> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role project with createdb login password 'projectpassword'; CREATE ROLE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap deploy:check :and you should get a nice green list of successes. ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 95bfb48319e2cc4caad35dab53269fd1c1385538 2587 2586 2014-01-10T21:07:32Z Neil 2 /* Create the production database */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld <Directory /var/www/project.domain.tld/public> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap deploy:check :and you should get a nice green list of successes. ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 8453530918bdfe4b393c01e81f920b850bc80d94 2589 2587 2014-01-10T23:34:50Z Neil 2 /* One-time setup: create the deploy user */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld <Directory /var/www/project.domain.tld/public> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap deploy:check :and you should get a nice green list of successes. ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 04cb9a6b8b48c8739183076af267427a824686ae 2590 2589 2014-01-10T23:35:19Z Neil 2 /* Create the Apache2 virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap deploy:check :and you should get a nice green list of successes. ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] c1a1887effe07aae9be416762e724e288398b5c6 2591 2590 2014-01-11T19:43:44Z Neil 2 /* Create the Apache2 virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap deploy:check :and you should get a nice green list of successes. ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] d7708607df7be176091deb6f62595952ec198a76 2593 2591 2014-01-12T15:30:52Z Neil 2 /* Create the Apache2 virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/feedcatcher.njae.me.uk.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap deploy:check :and you should get a nice green list of successes. ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 9f537b78ace57da53e2633e527da42d961b2a0b8 2594 2593 2014-01-12T15:31:07Z Neil 2 /* Create the Apache2 virtual host */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/feedcatcher.njae.me.uk' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap deploy:check :and you should get a nice green list of successes. ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 3f6e7086b64d61a46655526fa374fa76b4c83e18 2595 2594 2014-01-13T09:09:33Z Neil 2 /* Capify the project */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/project.domain.tld' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml config/initializers/secret_token.rb} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap deploy:check :and you should get a nice green list of successes. ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 289475691c3c024542602adc6c412fca42c1b199 2596 2595 2014-01-13T09:13:39Z Neil 2 /* Make the first deployment */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/project.domain.tld' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml config/initializers/secret_token.rb} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :and you should get a nice green list of successes. * Deploy the project! user@desktop:~project$ bundle exec cap production deploy ==Subsequent deployments== ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] d5fac575587ff362217cc72d7b587d09b4ccad26 2597 2596 2014-01-13T09:14:06Z Neil 2 /* Subsequent deployments */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/project.domain.tld' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml config/initializers/secret_token.rb} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :and you should get a nice green list of successes. * Deploy the project! user@desktop:~project$ bundle exec cap production deploy ==Subsequent deployments== They're as simple as the first deployment: user@desktop:~project$ bundle exec cap production deploy ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 499055519443cd378a3f76b373db1459f4ab394a 2598 2597 2014-01-13T09:15:36Z Neil 2 /* See also */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/project.domain.tld' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml config/initializers/secret_token.rb} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :and you should get a nice green list of successes. * Deploy the project! user@desktop:~project$ bundle exec cap production deploy ==Subsequent deployments== They're as simple as the first deployment: user@desktop:~project$ bundle exec cap production deploy ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * [http://www.modrails.com/documentation/Users%20guide%20Apache.html Phusion Passenger user guide] * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 9ec4485fa9d328b28069ea85cce813831d6c4acd 2599 2598 2014-01-13T09:19:45Z Neil 2 /* One-time setup: create the deploy user */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Check <code>deploy</code> is using the correct Ruby version deploy@server:~$ source /etc/profile.d/rvm.sh deploy@server:~$ rvm list rvm rubies =* ruby-2.1.0 [ x86_64 ] # => - current # =* - current && default # * - default deploy@server:~$ which ruby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/project.domain.tld' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml config/initializers/secret_token.rb} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :and you should get a nice green list of successes. * Deploy the project! user@desktop:~project$ bundle exec cap production deploy ==Subsequent deployments== They're as simple as the first deployment: user@desktop:~project$ bundle exec cap production deploy ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * [http://www.modrails.com/documentation/Users%20guide%20Apache.html Phusion Passenger user guide] * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 3e3ee405bc95a3f0c99e95aa5876c74dd59f70c8 2600 2599 2014-01-13T09:35:52Z Neil 2 /* See also */ wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Check <code>deploy</code> is using the correct Ruby version deploy@server:~$ source /etc/profile.d/rvm.sh deploy@server:~$ rvm list rvm rubies =* ruby-2.1.0 [ x86_64 ] # => - current # =* - current && default # * - default deploy@server:~$ which ruby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/project.domain.tld' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml config/initializers/secret_token.rb} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :and you should get a nice green list of successes. * Deploy the project! user@desktop:~project$ bundle exec cap production deploy ==Subsequent deployments== They're as simple as the first deployment: user@desktop:~project$ bundle exec cap production deploy ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * [http://www.modrails.com/documentation/Users%20guide%20Apache.html Phusion Passenger user guide] * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] * [http://jondavidjohn.com/blog/2013/06/capistrano-output-cleanup-with-steps How to simplify Capistrano output] == Bare bones deployment: Apache2 and mod_ruby, application at root of virtual host == This is a very basic installation simply to check that Rails is running properly. [http://www.howtoforge.com/ruby_on_rails_debian_etch_p2 See some config files for Apache] and [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ some more Apache2 config files]. * Create a directory to contain the application root@server:~# mkdir /var/www/project.domain.tld root@server:~# cd /var/www/project.domain.tld * Download the Rails app from the Suvbersion repository root@server:/var/www/project.domain.tld# svn checkout svn://svn.server.tld/path/to/app/ :Note that this will insert the last directory into the checked-out path, so the Rails root will be in <code>/var/www/project.domain.tld/app/</code> * Update <code>config/database.yml</code> production: adapter: mysql encoding: utf8 database: project_production pool: 5 username: project password: SecretPassword socket: /var/run/mysqld/mysqld.sock * Uncomment this line in <code>config/environment.yml</code> ENV['RAILS_ENV'] ||= 'production' * Create the database and its user root@server:~# mysql -u root -p mysql> create database project_production; mysql> create user 'project'@'localhost' identified by 'SecretPassword'; mysql> grant all on project_production.* to 'project'@'localhost'; * Run the database migration to create the tables and populate them root@server:/var/www/project.domain.tld# rake db:migrate RAILS_ENV=production * Create the Apache2 configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project./deployment-20090209/public ServerName depot.njae.me.uk <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/deployment-20090209/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi </Directory> RewriteEngine On # RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # In case Rails experiences terminal errors ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and reload it into Apache root@server:~# a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload == Initial deployment: Capistrano, Apache2, mod_ruby, application at root of virtual host == [http://www.capify.org/getting-started/rails Source instructions], [http://www.zorched.net/2008/06/17/capistrano-deploy-with-git-and-passenger/ Git + Passenger] This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * Modify <code>project/config/deploy.rb</code>: set :application, "project" set :repository, <nowiki>"http://svn.domain.tld/svn/repo/#{application}/branches/stable"</nowiki> set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end # As we're not running any other server processes (FastCGI, Mongrel, etc.), # we don't need to start and stop them. namespace :deploy do [:start, :stop, :restart].each do |t| desc "#{t} task is a no-op without other server processes" task t, :roles => :app do ; end end end :Note that we instruct Capistrano to copy <code>database.yml</code> into the right place, as it's not in the repository. Also, as we're not running any other server processes (FastCGI, Mongrel, etc.), we don't need to start and stop them. * Create the production database user's password and store it in the <code>production</code> stanza in <code>database.yml</code> * On the server, create the <code>/var/www/project.domain.tld</code> directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld * Copy across the <code>database.yml</code> file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * On the server, create the <code>/etc/apache2/sites-available/project.domain.tld</code> file: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/project.domain.tld/current/public> AllowOverride None Order allow,deny allow from all # General Apache options #AddHandler fastcgi-script .fcgi AddHandler cgi-script .cgi #AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI </Directory> RewriteEngine On # If the maintenance page exists, rewrite all requests to that page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.cgi [QSA,L] # RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 <nowiki>"<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :Note the <code>SetEnv</code> directive to ensure this is a production system. * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check :Fix any reported errors * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Apache2 and Mongrel cluster, deployed with Capistrano, application at root of virtual host == * Ensure Mongrel cluster is [[Ruby on Rails installation|installed on the server]]. === Get Mongrel running === * For testing, open ports in the [[Firewall setup|firewall]] to allow direct access to the Mongrel cluster. Modify <code>/etc/iptables.rules</code> to include these lines near the end: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart * Deploy the application to the server, for instance using Capistrano (as above). * On the server, <code>cd</code> to the root of the Rails application and set up the Mongrel cluster user@server:~$ cd /var/www/project.domain.tld/current user@server:current$ mongrel_rails cluster::configure -e production -p 3000 -N 3 Writing configuration file to config/mongrel_cluster.yml. * Modify the file <code>config/mongrel_cluster.yml</code> --- cwd : /var/www/depot.njae.me.uk/current log_file: log/mongrel.log port: "3000" environment: production pid_file: tmp/pids/mongrel.pid servers: 3 docroot: public user: bob group: bob :the <code>cwd</code>, <code>docroot</code>, <code>user</code>, and <code>group</code> elements are new. Bob is the name of a non-privileged user. * Start the cluster user@server:current$ mongrel_rails cluster::start starting port 3000 starting port 3001 starting port 3002 * Test the Mongrel cluster by pointing a web browser at server.domain.tld:3000, server.domain.tld:3001, and server.domain.tld:3002 === Make Apache a proxy for Mongrel === Now to configure Apache to act as a load-balancing proxy * Enable the modules Apache will need root@server:~# a2enmod proxy_balancer root@server:~# a2enmod proxy_http root@server:~# a2enmod rewrite * Edit the <code>/etc/apache2/sites-enabled/project.domain.tld</code> file <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/project.domain.tld/current/public ServerName project.domain.tld SetEnv RAILS_ENV production <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options FollowSymLinks MultiViews AllowOverride None Order allow,deny allow from all </Directory> <Directory /var/www/depot.njae.me.uk/current/public> # General Apache options Options +FollowSymLinks +ExecCGI AllowOverride None Order allow,deny allow from all </Directory> <Proxy *> Order allow,deny Allow from all </Proxy> <Proxy balancer://mongrel_cluster> <nowiki>BalancerMember http://127.0.0.1:3000</nowiki> <nowiki>BalancerMember http://127.0.0.1:3001</nowiki> <nowiki>BalancerMember http://127.0.0.1:3002</nowiki> </Proxy> RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # Rewrite rule to check for the index page: Apache serves this directly RewriteRule ^/$ /index.html [QSA] # Rewrite rule for static pages: Apache serves these direclty RewriteRule ^([^.]+)$ $1.html [QSA] # If no other rules match, pass the request to the Mongrel cluster # Redirect all non-static requests to cluster RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L] ## Deflate served pages to improve speed over the network #AddOutputFilterByType DEFLATE text/html text/plain text/xml application/xml application/xhtml+xml text/javascript text/css #BrowserMatch ^Mozilla/4 gzip-only-text/html #BrowserMatch ^Mozilla/4\.0[678] no-gzip #BrowserMatch \\bMSIE !no-gzip !gzip-only-text/html ## Uncomment for deflate debugging #DeflateFilterNote Input input_info #DeflateFilterNote Output output_info #DeflateFilterNote Ratio ratio_info #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate #CustomLog /var/log/apache2/project_deflate.log deflate # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead <nowiki>ErrorDocument 500 "<h2>Application error</h2>Rails application failed to start properly"</nowiki> # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(if you use the Deflate module, remember to enable it (<code>a2enmod deflate</code>) and create an empty log file (<code>touch /var/log/apache2/project_deflate.log</code>) before restarting Apache. * Reload the Apache configuration root@server:~# /etc/init.d/apache2 force-reload * Close the Mongrel ports to the outside world: delete these lines from <code>/etc/iptables.rules</code>: ## Temporarily allow Mongrel clusters across the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3000 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3001 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 3002 -j ACCEPT :and restart the firewall root@server:~# /etc/init.d/iptables restart === Update the project's capfile === See the [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book for details on using Monit to start and stop Mongrel clusters. If you're running Mongrel cluster as a service outside Monit, you'll need to include some modifications to <code>config/deploy.rb</code>. Create the custom start, stop, and restart tasks as shown below: # Custom tasks for starting and restarting Mongrel cluster namespace :deploy do desc "start the mongrel cluster" task :start, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl start" end desc "stop the mongrel cluster" task :stop, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl stop" end desc "restart the mongrel cluster" task :restart, :roles => :app do sudo "/usr/bin/mongrel_cluster_ctl restart" end end === Make Mongrel run as a service === * Create the file <code>/etc/mongrel/project.conf</code> # The user and group which run Mongrel user: bob group: bob # The location of the Rails application and the environment to run in cwd: /var/www/depot.njae.me.uk/current environment: production # The number of Mongrels in the cluster servers: 3 # The starting port port: "3000" # The IP addresses allowed to connect to Mongrel address: 127.0.0.1 # The location of the process ID files relative to the directory above pid_file: tmp/pids/mongrel.pid * Stop the existing Mongrel cluster and restart it with this new config file user@server:current$ mongrel_rails cluster::start user@server:current$ mongrel_cluster_ctl start * Remove the file <code>config/mongrel_cluster.yml</code> * Check you can still use the application. This checks that the config file is correct. * Stop the Mongrel cluster: user@server:current$ mongrel_cluster_ctl stop * Copy the Mongrel init.d script across root@server:~# cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/ * Modify the <code>/etc/init.d/mongrel_cluster</code> script to remove the reference to the non-existent <code>mongrel</code> user. Replace the reference to one to <code>root</code>. Chanage the lines near the top of the script to look like this: # USER=mongrel USER=root * Set up the service and start it root@server:~# chmod +x /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster defaults If you want to stop the Mongrel cluster running as a service, remove the init scripts: root@server:~# rm /etc/init.d/mongrel_cluster root@server:~# update-rc.d mongrel_cluster remove The [http://www.pragprog.com/titles/fr_deploy/deploying-rails-applications Deploying Rails Applications] book goes into more detail about using Monit to monitor the Mongrel cluster and restart Mongrel instances if any of them get into trouble. I've not got round to doing that yet. I will do if I decide to use Mongrel as my main production server. == Apache2 and Phusion Passenger, deployed with Capistrano, application at root of virtual host == Passenger is an application server for Rails that is run via Apache. The Passenger application server spawns a new process when it's needed and keeps it running for a while. After about 10 minutes without use, the application server process terminates to save system resources. When the next Rails request comes along, Apache spawns a new Passenger process to handle it. The nice thing is that the configuration is just about non-existent. This assumes the app will be deployed to the root of a virtual host all of its own. * Copy the project to a stable branch, for deployment. user@desktop:~/project$ svn copy -m "Creating stable branch" <nowiki>http://svn.domain.tld/svn/repo/project/trunk</nowiki> <nowiki>http://svn.domain.tld/svn/repo/project/branches/stable</nowiki> :(note the absence of trailing '/' characters in the SVN command) * Capify the project: user@desktop:~/project$ capify . * On the server, install the Passenger gem and install it: root@desktop:~# gem install passenger root@desktop:~# passenger-install-apache2-module * Add these lines to <code>/etc/apache2/httpd.conf</code> # Load the Passenger module for Rails applications LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6/ext/apache2/mod_passenger.so PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.6 PassengerRuby /usr/bin/ruby1.8 * On the server, create the /var/www/project.domain.tld directory, and make it writeable by the user: root@server:~# mkdir /var/www/project.domain.tld root@server:~# chown -R user:user /var/www/project.domain.tld :Note that Passenger runs as the user who owns the <code>config/environment.rb</code> file, so ensure that user has adequate rights to this bit of the file system. * Copy across the database.yml file user@desktop:~/project$ ssh user@server 'mkdir /var/www/project.domain.tld/config' user@desktop:~/project$ scp config/database.yml \ user@server:/var/www/project.domain.tld/config/database.yml * Create the production database on the server root@server:~# mysql -u root -p mysql> create database project_production; mysql> grant all on project_production.* to 'project'@'localhost' identified by 'password'; mysql> quit; * Create the virtual host configuration file, <code>/etc/apache2/sites-available/project.domain.tld</code> containing just this: <VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public RewriteEngine On # If the system maintenance page exists, serve that instead of any other page RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f RewriteCond %{SCRIPT_FILENAME} !maintenance.html RewriteRule ^.*$ /system/maintenance.html [L] # In case Rails experiences terminal errors # Instead of displaying this message you can supply a file here which will be rendered instead ErrorDocument 500 "<nowiki><h2>Application error</h2>Rails application failed to start properly</nowiki>" # this not only blocks access to .svn directories, but makes it appear # as though they aren't even there, not just that they are forbidden <DirectoryMatch "^/.*/\.svn/"> ErrorDocument 403 /404.html Order allow,deny Deny from all Satisfy All </DirectoryMatch> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Enable the site and restart Apache root@server:~# /etc/init.d/a2ensite project.domain.tld root@server:~# /etc/init.d/apache2 reload * Set up and check the deployment user@desktop:~/project$ cap deploy:setup user@desktop:~/project$ cap deploy:check * For deployment, you need to modify the Capistrano deployment recipe, <code>project/config/deploy.rb</code>. Update the <code>:start</code>. <code>:stop</code>. and <code>:restart</code> tasks to be as shown: set :application, "project" set :repository, "<nowiki>http://svn.domain.tld/svn/repo/#{application}/branches/stable</nowiki>" set :deploy_to, "/var/www/project.domain.tld/" set :scm, :subversion set :scm_username, 'developer_user' role :app, "project.domain.tld" role :web, "project.domain.tld" role :db, "project.domain.tld", :primary => true # Copy the database.yml file across, as it's not kept in the SVN repository after "deploy:update_code" , :configure_database desc "copy database.yml into the current release path" task :configure_database, :roles => :app do db_config = "#{deploy_to}/config/database.yml" run "cp #{db_config} #{release_path}/config/database.yml" end namespace :deploy do task :start, :roles => :app do end task :stop, :roles => :app do end desc "Restart Application" task :restart, :roles => :app do run "touch #{release_path}/tmp/restart.txt" end end * Make the first deployment user@desktop:~/project$ cap deploy:migirations == Subsequent deployments with Capistrano from a stable branch == Assume a bunch of enhancements have been made in the trunk. These need to be copied to the stable branch in Subversion and redeployed. * Check out a working copy of the stable branch user@desktop:~$ mkdir -p /path/to/stable/branch/working/copy user@desktop:~$ cd /path/to/stable/branch/working/copy user@desktop:copy$ svn checkout uri://repository/app/branches/stable user@desktop:copy$ cd stable * Find the latest revision number of the stable branch user@desktop:stable$ svn log :note revision number of last commit. Assume its revision 30. * Find the latest revision number of the trunk. Either look at the repository, or update the trunk and look at the log user@desktop:trunk$ svn update user@desktop:trunk$ svn log :Note the different directory: do this in a different console. Assume the trunk is at revision 45. * Merge the changed from the trunk into the stable branch user@desktop:stable$ svn merge -r 30:45 uri://repository/app/trunk * Commit the updated stable branch to the repository user@desktop:stable$ svn commit -m "Merged updates from trunk into stable branch" * Redeploy the application user@desktop:stable$ cap deploy:migrations == See also == * [http://wiki.rubyonrails.org/deployment/apache-mongrel Rails Wiki: Apache + Mongrel] * [http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/ Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You] * [http://www.kodefoo.com/2007/2/18/deploying-rails-on-ubuntu-dapper/ Deploying Rails on Ubuntu Dapper] * [http://mongrel.rubyforge.org/wiki/Apache Mongrel docs on Apache] (out of date) * [http://rails.aizatto.com/2007/05/20/deploying-ruby-on-rails-on-ubuntu-feisty-fawn-via-mongrel-cluster-and-apache/ Deploying Ruby on Rails on Ubuntu Feisty Fawn via Mongrel Cluster and Apache] * [http://www.modrails.com/documentation/Users%20guide.html Passenger user guide] * [http://www.railsgarden.com/2008/04/12/configurating-passenger-mod_rails-on-slicehost-with-ubuntu-710/ How to deploy Passenger on Ubuntu] [[Category:Ruby]] [[Category:Rails]] 0565be53980bbd97eebdc21366b92beee4f3684d 2602 2600 2014-01-13T09:47:54Z Neil 2 wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Check <code>deploy</code> is using the correct Ruby version deploy@server:~$ source /etc/profile.d/rvm.sh deploy@server:~$ rvm list rvm rubies =* ruby-2.1.0 [ x86_64 ] # => - current # =* - current && default # * - default deploy@server:~$ which ruby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/project.domain.tld' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml config/initializers/secret_token.rb} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :and you should get a nice green list of successes. * Deploy the project! user@desktop:~project$ bundle exec cap production deploy ==Subsequent deployments== They're as simple as the first deployment: user@desktop:~project$ bundle exec cap production deploy ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * [http://www.modrails.com/documentation/Users%20guide%20Apache.html Phusion Passenger user guide] * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] * [http://jondavidjohn.com/blog/2013/06/capistrano-output-cleanup-with-steps How to simplify Capistrano output] [[Category:Ruby]] [[Category:Rails]] 1e0e72682b1974acfe1d9202b3cf27eb1c62b005 0 1543 2564 2200 2014-01-06T22:40:57Z Neil 2 /* Iptables.rules */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. == Remove ufw == The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. == Add a bespoke iptables firewall == * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 8375d4f6a906baea8dd66bf46b659625d282e1c0 0 1526 2588 2546 2014-01-10T22:05:10Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] c0a1890aa4b54c8b8dd5ef3808b7b49aa49df129 2592 2588 2014-01-12T15:26:01Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] b443c0908d72067ddd9f3f2f6c04a527e00dd17e 0 1545 2604 2436 2014-01-13T09:51:51Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server mysql-client == Passwords == The first thing is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <code>host_name</code> is the name of this host, e.g. <code>server</code>. :(seemingly not needed for Ubuntu 10.04) * Add passwords to the <code>root</code> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'127.0.0.1' = PASSWORD('newpwd'); == Remote access == If you want to access MySQL databases from a remote machine, you'll need to have MySQL accept connections across the network. Edit <code>/etc/mysql/my.cnf</code> and change the <code>bind-address</code> entry to be the IP of the machine: bind-address = 192.168.2.252 The restart MySQL: root@server:~# /etc/init.d/mysql restart Note that the [[Firewall setup|firewall]] is configured to only allow connections to MySQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] 45c23d0227c2394055b3725522e3ed65bf255d85 0 1631 2605 2561 2014-01-13T09:52:31Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client pgadmin3 == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf</code> and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/9.1/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] ef9980fabcd50a3369eea3c3d490bcd4b5181dc3 0 1533 2606 2466 2014-01-13T09:52:52Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# service apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <cocde>/etc/apache2/ssl/certs</code>. root@server:~# mkdir -p /etc/apache2/ssl/private root@server:~# mkdir -p /etc/apache2/ssl/certs root@server:~# cp /etc/ssl/njae/certs/squirrelmail-cert.pem /etc/apache2/ssl/certs/ root@server:~# cp /etc/ssl/njae/private/squirrelmail-key.insecure.pem /etc/apache2/ssl/private/ Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. root@server:~# chown -R www-data:www-data /etc/apache2/ssl/ root@server:~# chmod -R 640 /etc/apache2/ssl/ Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/private/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 61747f24ba90b3d7421f941e4164ca3b57a771eb 0 1515 2607 2524 2014-01-13T09:53:51Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 12.04 LTS and 13.10 (adapted from 8.04 LTS by way of 10.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 0a41dd5ed294ea79a39b79ad4182120e531bc2d5 0 1526 2608 2592 2014-01-20T09:18:49Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. [[Category:Server setup]] bca43c96a111f8d6d6b6da82cb2dd155fbcdff18 2609 2608 2014-01-20T09:30:16Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. [[Category:Server setup]] 440c7a864e670c66ef78ef387e6a904057fbc539 2610 2609 2014-01-20T09:30:32Z Neil 2 /* Install Sublime Text */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. [[Category:Server setup]] 9c48285ac7042cfeea47effbcfe4a25f6ba006da 2611 2610 2014-01-20T09:33:02Z Neil 2 /* Install Sublime Text */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 3a765284439b4d6132e73f637e851cb3bdccd8dc 2613 2611 2014-01-20T15:22:07Z Neil 2 /* Install Sublime Text */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 8ce4ffae7f97d78ebed2c93d7f44404651a32e41 2614 2613 2014-01-27T09:19:22Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 42d2835f960b01d059f0a716f79d34f8145866ce 2615 2614 2014-01-27T11:40:55Z Neil 2 /* Starting a new Ruby project */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper user@desktop:~$ sudo pip3 install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python3' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv p3basic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon p3basic (p3basic)user@desktop:~$ deactivate == Get Akonadi (and Kontact) to start == Akonadi is a back-end for managing the metadata handled by Kontact. At present (Ubuntu 10.04) it only handles the contact list. Unfortunately, it doesn't work very well. To get it working, * Install the required packages, remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ sudo apt-get install virtuoso-server mysql-server-5.1 libakonadi-kcal-next4 \ libsmokeakanodi3 akonadi-kde-resource-googledata libakanodi-next4 akonadiconsole * Add an Akonadi resource to KDE. It's on System Settings -> Advanced -> KDE Resources, and add an 'Akonadi Address book', and set it as standard (you may have needed to get Akanodi running first and created a personal address book in it already). * Remove the existing MySQL database it uses, create a new one, and restart it. user@desktop:~$ rm -r $HOME/.local/share/akonadi user@desktop:~$ akonadictl start user@desktop:~$ mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ user@desktop:~$ mysql_upgrade --socket=$HOME/.local/share/akonadi/db_misc/mysql.socket user@desktop:~$ akonadictl stop user@desktop:~$ akonadictl start :Ignore any error messages emitted by MySQL during its steps. * Set Akonadi to autostart in KDE. In System Settings -> Advanced -> Autostart, add the script <code>/usr/bin/akonadictl start</code> Some people have reported that moving the global config file to the per-user area can work: user@desktop:~$ cp /etc/akonadi/mysql-global.conf ~/.config/akonadi/mysql-local.conf (its in <code>/usr/local/share/akonadi</code> on non-Debian-derived boxes, apparently). If you do that and it still doesn't work, you may have luck with uncommenting one or other of the <code>sql_mode=</code> lines in that file. Restart Kontact a few times, especially if you've mucked around with trying to get the contact lists working. You may need to restart Kontact after booting anyway, to make it use Akonadi properly. (Details from the [http://userbase.kde.org/Akonadi_4.4/Troubleshooting#Kubuntu_10.4_Upgrade Akonadi troubleshooting guide].) == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 9b1a6b3352fb98b3056979cd351f5773cb8e5aca 2616 2615 2014-01-27T11:48:30Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key and set AltGr key as Mode_swich== In the first instance, the KDE Keyboard settings allow you to define the Compose key directly. Do that first, and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 3d18b4d39075ab75f59702eafed58cca41d0bda7 2619 2616 2014-02-01T21:55:26Z Neil 2 /* Enable the Compose key and set AltGr key as Mode_swich */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 2f43b811fc4193de2ac1049404c8d9ee0045842a 2620 2619 2014-02-14T10:26:23Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install python3-dev build-essential python3-tk tk-dev libpng12-dev python3-numpy python3-scipy python3-pandas pandoc user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install Scikit-learn user@desktop:~$ sudo pip install scikit-learn user@desktop:~$ sudo pip3 install scikit-learn * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 20b0e95af70da8c5c62c497b74854c332bb8ba97 2621 2620 2014-02-15T22:44:28Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install Scikit-learn user@desktop:~$ sudo pip install scikit-learn user@desktop:~$ sudo pip3 install scikit-learn * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] d914430de0544bef20bc57ee132137fe6d5bb6c3 0 1593 2612 2504 2014-01-20T10:56:18Z Neil 2 /* Setup gitweb */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create it in <code>gitolite-admin</code> (and push the changes), then clone it on the client machine. ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 70b764592f2c7b8903b5c134bf5fc9030eafa13b 0 1544 2617 2523 2014-01-27T19:00:06Z Neil 2 /* Getting the servers running */ Added note about turning off dnssec wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <code>/etc/bind</code> * Alter the <code>/etc/bind/named.conf.options</code> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; dnssec-validation auto; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; :if you have problems with your forwarders not implementing DNSSEC (<code>error (insecurity proof failed) resolving './NS/IN'</code> appearing in <code>/var/log/syslog</code> and intermittent DNS service), you may want to replace the dnssec lines with: // dnssec-validation auto; dnssec-enable no; dnssec-validation no; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <code>/etc/bind/named.conf.local</code> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <code>/var/cache/bind</code> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <code>/etc/bind/</code>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <code>/etc/bind/db.domain.tld</code> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <code>/etc/bind/db.1.168.192</code> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update <code>/etc/resolv.conf</code> to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 192.168.2.251 nameserver 192.168.2.252 :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 57939bbc6602a23789e5b69af5d4c476f84613e6 2622 2617 2014-02-22T16:45:34Z Neil 2 /* Update the router */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <code>/etc/bind</code> * Alter the <code>/etc/bind/named.conf.options</code> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; dnssec-validation auto; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; :if you have problems with your forwarders not implementing DNSSEC (<code>error (insecurity proof failed) resolving './NS/IN'</code> appearing in <code>/var/log/syslog</code> and intermittent DNS service), you may want to replace the dnssec lines with: // dnssec-validation auto; dnssec-enable no; dnssec-validation no; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <code>/etc/bind/named.conf.local</code> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <code>/var/cache/bind</code> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <code>/etc/bind/</code>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <code>/etc/bind/db.domain.tld</code> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <code>/etc/bind/db.1.168.192</code> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update <code>/etc/resolv.conf</code> to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 192.168.2.251 nameserver 192.168.2.252 :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. For DD-WRT, the settings to use are: * Router IP: Local DNS: 192.168.0.252 == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <tt>/usr/share/doc/bind9/README.Debian.gz</tt> file on your machine's local disk has some good pointers. [[Category:Server setup]] 2b46a27409f109f02da6cff4f784bc1ec9489188 2623 2622 2014-02-22T16:49:11Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# apt-get install bind9 bind9-doc :This will install the configuration files in <code>/etc/bind</code> * Alter the <code>/etc/bind/named.conf.options</code> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; dnssec-validation auto; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; :if you have problems with your forwarders not implementing DNSSEC (<code>error (insecurity proof failed) resolving './NS/IN'</code> appearing in <code>/var/log/syslog</code> and intermittent DNS service), you may want to replace the dnssec lines with: // dnssec-validation auto; dnssec-enable no; dnssec-validation no; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <code>/etc/bind/named.conf.local</code> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <code>/var/cache/bind</code> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <code>/etc/bind/</code>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <code>/etc/bind/db.domain.tld</code> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <code>/etc/bind/db.1.168.192</code> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update <code>/etc/resolv.conf</code> to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 192.168.1.251 nameserver 192.168.1.252 :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Restart bind: root@server:~# /etc/init.d/bind restart :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. For DD-WRT, the settings to use are: * '''Router IP''' ** ''Local DNS:'' 192.168.0.252 * '''Network Address Server Settings (DHCP)''' ** ''Static DNS 1:'' 192.168.0.252 ** ''Static DNS 2:'' 192.168.0.251 ** ''Static DNS 3:'' 0.0.0.0 ** ''WINS:'' 0.0.0.0 ** ''Use DNSMasq for DHCP:'' OFF ** ''Use DNSMasq for DNS:'' OFF ** ''DHCP-Authoritative:'' OFF == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <code>/usr/share/doc/bind9/README.Debian.gz</code> file on your machine's local disk has some good pointers. [[Category:Server setup]] 4cebef2b18818a66f2b1061959a16cfaf873b8e3 0 1579 2618 2602 2014-01-29T11:27:02Z Neil 2 /* Create the Apache2 virtual host */ Updated PassengerRuby config wikitext text/x-wiki Various different ways to deploy a Ruby on Rails application. Assumes the [[Ruby on Rails installation|server is set up]] and the [[Ruby on Rails project creation#Production_database|databases are created]]. Note that <code>config/database.yml</code>, <code>config/initializers/secret_token.rb</code>, and <code>config/deploy.rb</code> aren't in the the Git repo. ==One-time setup: create the deploy user== * On the server where the production systems will be deployed, create a new <code>deploy</code> user and lock the account root@server:~# adduser deploy root@server:~# passwd -l deploy Add the deploy user to the [[Protect_SSH#Changes_to_config_file|list of users who can SSH into the server]]. Add deploy to the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code> AllowUsers user1 user2 user3 git deploy :the restart sshd: root@server:~# service ssh restart * Import the public keys from all users that need to trigger a deployment. root@server:~# sudo -u deploy -i deploy@server:~$ scp user@desktop:.ssh/id_rsa.pub user.pub deploy@server:~$ cat user.pub >> .ssh/authorized_keys deploy@server:~$ rm user.pub :<code>user@desktop</code> should now be able to log into <code>deploy@server<code> without a password. * Generate a public key for <code>deploy@server</code>: deploy@server:~$ ssh-keygen -t rsa * Go back to wherever the [[Git setup|Gitolite admin]] repository is. Add the <code>deploy</code> user's public key to the gitolite admin repo. user@desktop:/gitolite-admin$ scp deploy@server:.ssh/id_rsa.pub keydir/deploy.pub * In <code>conf/gitolite.conf</code>, add <code>deploy</code> as a read-only user for the repos you want them to deploy, e.g. repo sample RW+ = user R = daemon deploy desc = Sample Rails application * Commit and push the changes. user@desktop:/gitolite-admin$ git add . user@desktop:/gitolite-admin$ git commit -a -m "Added deploy user" user@desktop:/gitolite-admin$ git push * Back as the <code>deploy</code> user, check that you can clone a repo. (This also allows you to pass SSH's host authenticity test.) deploy@ogedei:~$ git clone git@git.domain.tld:sample.git * Make sure the <code>deploy</code> user can use RVM. Add the following to <code>/home/deploy/.bashrc</code>: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Check <code>deploy</code> is using the correct Ruby version deploy@server:~$ source /etc/profile.d/rvm.sh deploy@server:~$ rvm list rvm rubies =* ruby-2.1.0 [ x86_64 ] # => - current # =* - current && default # * - default deploy@server:~$ which ruby /usr/local/rvm/rubies/ruby-2.1.0/bin/ruby ==Per-project setup== These are the steps needed to set up a project for the initial deployment. ===Capify the project=== * On the development machine, initialise Capistrano: user@desktop:~/project$ cap install STAGES=production :(<code>STAGES=production</code> omits the default "staging" deploy target.) * Edit <code>project/config/deploy/production.rb</code> role :app, %w{deploy@server.domain.tld} role :web, %w{deploy@server.domain.tld} role :db, %w{deploy@server.domain.tld}, primary: true :and ensure the "exteneded server syntax" line further down the file is commented out. * Edit <code>project/config/deploy.rb</code> set :application, 'project' set :repo_url, 'git@git.domain.tld:project.git' set :branch, 'master' set :default_stage, 'production' set :deploy_to, '/var/www/project.domain.tld' set :scm, :git set :format, :pretty # set :log_level, :debug # set :pty, true set :linked_files, %w{config/database.yml config/initializers/secret_token.rb} set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system} set :default_env, { path: "/opt/ruby/bin:$PATH" } set :keep_releases, 5 namespace :deploy do desc 'Restart application' task :restart do on roles(:app), in: :sequence, wait: 5 do execute :touch, release_path.join('tmp/restart.txt') end end after :restart, :clear_cache do on roles(:web), in: :groups, limit: 3, wait: 10 do # Here we can do anything such as: # within release_path do # execute :rake, 'cache:clear' # end end end after :finishing, 'deploy:cleanup' end * Edit <code>Capfile</code> require 'capistrano/rvm' require 'capistrano/bundler' require 'capistrano/rails/assets' require 'capistrano/rails/migrations' ===Create the Apache2 virtual host=== * Find out the correct path to use for the ruby you want (assumed to to be Ruby 2.1.0 here): root@ogedei:~# rvm use 2.1.0 root@ogedei:~# passenger-config --ruby-command passenger-config was invoked through the following Ruby interpreter: Command: /usr/local/rvm/gems/ruby-2.1.0/wrappers/ruby Version: ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-linux] To use in Apache: PassengerRuby /usr/local/rvm/gems/ruby-2.1.0/wrappers/ruby To use in Nginx : passenger_ruby /usr/local/rvm/gems/ruby-2.1.0/wrappers/ruby To use with Standalone: /usr/local/rvm/gems/ruby-2.1.0/wrappers/ruby /usr/bin/passenger start :and place the <code> PassengerRuby</code> path in the Apache2 config below. * Create the site virtual host file for Apache2. Create the file <code>/etc/apache2/site-available/project.domain.tld</code>: VirtualHost *:80> ServerName project.domain.tld DocumentRoot /var/www/project.domain.tld/current/public PassengerRuby /usr/local/rvm/gems/ruby-2.1.0/wrappers/ruby <Directory /var/www/project.domain.tld/current/public/> Allow from all Options -MultiViews </Directory> LogLevel warn ErrorLog /var/log/apache2/error.log CustomLog /var/log/apache2/project.access.log combined ServerSignature Off </VirtualHost> * Create the directories and give them the correct permissions: <code>deploy</code> need to write to them, <code>www-data</code> needs to read them: root@ogedei:~# mkdir -p /var/www/project.domain.tld/public root@ogedei:~# mkdir -p /var/www/project.domain.tld/config root@ogedei:~# chown -R deploy:www-data /var/www/project.domain.tld root@ogedei:~# ls -lah /var/www/project.domain.tld/ /var/www/project.domain.tld/: total 16 drwxr-xr-x 4 deploy www-data 4.0K Jan 10 20:35 . drwxr-xr-x 13 root root 4.0K Jan 10 20:35 .. drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 config drwxr-xr-x 2 deploy www-data 4.0K Jan 10 20:35 public ===Create the production database=== * On the server, create the (empty) database root@server:~# sudo -u postgres psql postgres could not change directory to "/root" psql (9.1.11) Type "help" for help. postgres=# create role projectuser with createdb login password 'projectpassword'; CREATE ROLE postgres=# create database projectdb owner=projectuser; CREATE DATABASE postgres=# \q * Add the production database password to <code>config/database.yml</code>. ===Make the first deployment=== * Run the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :which will create additional directories on the server. * Manually copy across the <code>conig/database.yml</code> file to the server: user@desktop:~project$ scp config/database.yml deploy@server:/var/www/project.domain.tld/shared/config/ * Rerun the deployment check: user@desktop:~project$ bundle exec cap production deploy:check :and you should get a nice green list of successes. * Deploy the project! user@desktop:~project$ bundle exec cap production deploy ==Subsequent deployments== They're as simple as the first deployment: user@desktop:~project$ bundle exec cap production deploy ==See also== * [http://capistranorb.com/documentation/getting-started/installation/ Capistrano installation] * [https://github.com/capistrano/capistrano/blob/master/README.md Capistrano readme] * [https://github.com/capistrano/capistrano/wiki/Capistrano-Tasks Capistrano tasks] (also available with the command <code>bundle exec cap -T</code> * [http://www.modrails.com/documentation/Users%20guide%20Apache.html Phusion Passenger user guide] * Maintenance page: a [https://github.com/capistrano/maintenance Capistrano maintenance] gem and some [http://viget.com/extend/server-maintenance-mode-for-rails-capistrano-and-apache2instructions with Apache2 config] for using it. * An old but still useful [https://gist.github.com/jrochkind/2161449 guide to using Capistrano on a single box] * [http://jondavidjohn.com/blog/2013/06/capistrano-output-cleanup-with-steps How to simplify Capistrano output] [[Category:Ruby]] [[Category:Rails]] 9b32231c3ea6b0d90c74068e297cdcc5a7df12c8 6 1632 2624 2014-02-23T19:46:18Z Neil 2 From http://www.vacationhomes.net/blog/pub/Zhangjiajie-National-Forest-Park.jpg wikitext text/x-wiki From http://www.vacationhomes.net/blog/pub/Zhangjiajie-National-Forest-Park.jpg e5afd19f1c8f70f553df9b26a3ef9e6a8a84574d 2637 2624 2014-02-23T22:31:40Z Neil 2 wikitext text/x-wiki From http://www.vacationhomes.net/blog/pub/Zhangjiajie-National-Forest-Park.jpg [[Category:Star Wars: Edge of the Empire]] 118a915226e685473785252f13ae5e8d1afbdb87 0 1633 2625 2014-02-23T19:49:20Z Neil 2 Created page with "Some house rules for [[:Category:Star Wars: Edge of the Empire|Star Wars: Edge of the Empire]] [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie Nationa…" wikitext text/x-wiki Some house rules for [[:Category:Star Wars: Edge of the Empire|Star Wars: Edge of the Empire]] [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] I ran our first serious spaceship combat session today. The PCs were dropping off some ryll to some Geonosian buyers when four of Teemo the Hutt's cloakshape fighters dropped in. The deal was happening atop on column of a "forest" of limestone columns, similar to Zhangjiajie National Forest Park. Cue a dogfight between the crew of the Krayt Fang and a minion pair of cloakshapes, while dodging around the scenery! To help things along, I made a couple of changes while merging the combat and chase rules. ==Positioning, not speed== First of all, since everyone's dodging terrain, absolute speed isn't that important. What's important is how fast you're going for the conditions, and I use the ship's Speed rating to represent acceleration and power. At the start of every round, all pilots make a Positioning check as per pp. 240-1. This doesn't use up the pilot's Action for the round. Pilots declare their chosen speeds in turn, and then get a chance to increase (but not decrease) their speed once they've heard what their opponents are doing. High speed is good, because all pilot- and gunnery-related checks for the rest of the round get an extra boost die for every point of speed they're faster. Once you've bid, roll the Positioning check as per p. 240 and compare results with your opponent. You're after how many net successes and advantages each side gets when compared to the other. Net successes can be used to change the range of the encounter for the rest of the round. You can also spend a success to either upgrade or downgrade something in a test in the rest of the round. Advantages and disadvantages get spent as per p. 236. A Despair means you crash into something (p. 242). A Triumph means you can ignore the Navigation Hazard when rolling positioning in the next round. This change also means that the Accelerate/Declerate, Fly/Drive, and Punch It manoeuvres don't make any sense. ===Example:=== The Krayt Fang is being chased by two cloakshape fighters. The Krayt's pilot calls speed 2, the fighters call speed 4. The Krayt's pilot rolls GGY (skill) + K (handling) + RR (speed & silhouette) + KK (terrain), getting two failures and an advantage. The fighters roll GGY (skill) + RRPP (speed & silhouette) + KK (terrain), getting three failures and a disadvantage. Both sides are fairly inept, but the Krayt slightly less so. The Krayt's pilot can either change the range by one band, or apply an upgrade or downgrade to any one test this round. In addition, the Krayt's pilot spends effectively two advantages to impose a setback die on the fighters' next action. Because the fighters were faster, they get two boost dice to all pilot and gunnery checks for the rest of the round. ==See also== See me revised [[Gaining the Advantage]]. [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] e65188c8cf1b94ff6ff31657f9744d503ad72561 2627 2625 2014-02-23T19:51:32Z Neil 2 /* See also */ wikitext text/x-wiki Some house rules for [[:Category:Star Wars: Edge of the Empire|Star Wars: Edge of the Empire]] [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] I ran our first serious spaceship combat session today. The PCs were dropping off some ryll to some Geonosian buyers when four of Teemo the Hutt's cloakshape fighters dropped in. The deal was happening atop on column of a "forest" of limestone columns, similar to Zhangjiajie National Forest Park. Cue a dogfight between the crew of the Krayt Fang and a minion pair of cloakshapes, while dodging around the scenery! To help things along, I made a couple of changes while merging the combat and chase rules. ==Positioning, not speed== First of all, since everyone's dodging terrain, absolute speed isn't that important. What's important is how fast you're going for the conditions, and I use the ship's Speed rating to represent acceleration and power. At the start of every round, all pilots make a Positioning check as per pp. 240-1. This doesn't use up the pilot's Action for the round. Pilots declare their chosen speeds in turn, and then get a chance to increase (but not decrease) their speed once they've heard what their opponents are doing. High speed is good, because all pilot- and gunnery-related checks for the rest of the round get an extra boost die for every point of speed they're faster. Once you've bid, roll the Positioning check as per p. 240 and compare results with your opponent. You're after how many net successes and advantages each side gets when compared to the other. Net successes can be used to change the range of the encounter for the rest of the round. You can also spend a success to either upgrade or downgrade something in a test in the rest of the round. Advantages and disadvantages get spent as per p. 236. A Despair means you crash into something (p. 242). A Triumph means you can ignore the Navigation Hazard when rolling positioning in the next round. This change also means that the Accelerate/Declerate, Fly/Drive, and Punch It manoeuvres don't make any sense. ===Example:=== The Krayt Fang is being chased by two cloakshape fighters. The Krayt's pilot calls speed 2, the fighters call speed 4. The Krayt's pilot rolls GGY (skill) + K (handling) + RR (speed & silhouette) + KK (terrain), getting two failures and an advantage. The fighters roll GGY (skill) + RRPP (speed & silhouette) + KK (terrain), getting three failures and a disadvantage. Both sides are fairly inept, but the Krayt slightly less so. The Krayt's pilot can either change the range by one band, or apply an upgrade or downgrade to any one test this round. In addition, the Krayt's pilot spends effectively two advantages to impose a setback die on the fighters' next action. Because the fighters were faster, they get two boost dice to all pilot and gunnery checks for the rest of the round. ==See also== * My revised [[Gaining the Advantage]]. * A [http://community.fantasyflightgames.com/index.php?showtopic=99926 forum thread discussing this] [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] 26ab207ca81b003dc4b4f95f17ee3b7ab23147d0 2651 2627 2014-02-24T10:41:49Z Neil 2 wikitext text/x-wiki Some house rules for [[:Category:Star Wars: Edge of the Empire|Star Wars: Edge of the Empire]] [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] I ran our first serious spaceship combat session today. The PCs were dropping off some ryll to some Geonosian buyers when four of Teemo the Hutt's cloakshape fighters dropped in. The deal was happening atop on column of a "forest" of limestone columns, similar to Zhangjiajie National Forest Park. Cue a dogfight between the crew of the Krayt Fang and a minion pair of cloakshapes, while dodging around the scenery! To help things along, I made a couple of changes while merging the combat and chase rules. ==Positioning, not speed== First of all, since everyone's dodging terrain, absolute speed isn't that important. What's important is how fast you're going for the conditions, and I use the ship's Speed rating to represent acceleration and power. At the start of every round, all pilots make a Positioning check as per pp. 240-1. This doesn't use up the pilot's Action for the round. Pilots declare their chosen speeds in turn, and then get a chance to increase (but not decrease) their speed once they've heard what their opponents are doing. High speed is good, because all pilot- and gunnery-related checks for the rest of the round get an extra boost die for every point of speed they're faster. Once you've bid, roll the Positioning check as per p. 240 and compare results with your opponent. You're after how many net successes and advantages each side gets when compared to the other. Net successes can be used to change the range of the encounter for the rest of the round. You can also spend a success to either upgrade or downgrade something in a test in the rest of the round. Advantages and disadvantages get spent as per p. 236. A Despair means you crash into something (p. 242). A Triumph means you can ignore the Navigation Hazard when rolling positioning in the next round. This change also means that the Accelerate/Declerate, Fly/Drive, and Punch It manoeuvres don't make any sense. ===Example:=== The Krayt Fang is being chased by two cloakshape fighters. The Krayt's pilot calls speed 2, the fighters call speed 4. The Krayt's pilot rolls GGY (skill) + K (handling) + RR (speed & silhouette) + KK (terrain), getting two failures and an advantage. The fighters roll GGY (skill) + RRPP (speed & silhouette) + KK (terrain), getting three failures and a disadvantage. Both sides are fairly inept, but the Krayt slightly less so. The Krayt's pilot can either change the range by one band, or apply an upgrade or downgrade to any one test this round. In addition, the Krayt's pilot spends effectively two advantages to impose a setback die on the fighters' next action. Because the fighters were faster, they get two boost dice to all pilot and gunnery checks for the rest of the round. ==See also== * My revised [[Gain the Advantage]]. * A [http://community.fantasyflightgames.com/index.php?showtopic=99926 forum thread discussing this] [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] 71fb2f1459feb7406d6ee6a36f71a25e13245646 2652 2651 2014-02-24T10:58:24Z Neil 2 /* Positioning, not speed */ wikitext text/x-wiki Some house rules for [[:Category:Star Wars: Edge of the Empire|Star Wars: Edge of the Empire]] [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] I ran our first serious spaceship combat session today. The PCs were dropping off some ryll to some Geonosian buyers when four of Teemo the Hutt's cloakshape fighters dropped in. The deal was happening atop on column of a "forest" of limestone columns, similar to Zhangjiajie National Forest Park. Cue a dogfight between the crew of the Krayt Fang and a minion pair of cloakshapes, while dodging around the scenery! To help things along, I made a couple of changes while merging the combat and chase rules. ==Positioning, not speed== First of all, since everyone's dodging terrain, absolute speed isn't that important. What's important is how fast you're going for the conditions, and I use the ship's Speed rating to represent acceleration and power. At the start of every round, all pilots make a Positioning check as per pp. 240-1. This doesn't use up the pilot's Action for the round. Pilots declare their chosen speeds in turn, and then get a chance to increase (but not decrease) their speed once they've heard what their opponents are doing. Once you've bid, roll the Positioning check as per p. 240 and compare results with your opponent. You're after how many net successes and advantages each side gets when compared to the other. {|{{Prettytable}} align=right ! {{Hl3}}|'''Speed difference''' ! {{Hl3}}|'''Range bands moved''' |- | 0-1 || Range remains unchanged |- | 2 || Move between close and short range |- | 4 || Move between short and medium range |} Net successes add to your speed if you want to change the range of the encounter for the rest of the round, using costs to the right. You can also spend a success to either upgrade or downgrade something in a test in the rest of the round. Advantages and disadvantages get spent as per p. 236. A Despair means you crash into something (p. 242). A Triumph means you can ignore the Navigation Hazard when rolling positioning in the next round. This change also means that the Accelerate/Declerate, Fly/Drive, and Punch It manoeuvres don't make any sense. ===Example:=== The Krayt Fang is being chased by two cloakshape fighters. The Krayt's pilot calls speed 2, the fighters call speed 4. The Krayt's pilot rolls GGY (skill) + K (handling) + RR (speed & silhouette) + KK (terrain), getting two failures and an advantage. The fighters roll GGY (skill) + RRPP (speed & silhouette) + KK (terrain), getting three failures and a disadvantage. Because the fighters were faster, they can change the range from close to short range. Both sides are fairly inept, but the Krayt slightly less so. The Krayt's pilot can spend the success to effectively increase the Krayt's speed, cancelling the range change. Alternatively, she can keep it to apply an upgrade or downgrade to any one test this round. In addition, the Krayt's pilot spends effectively two advantages to impose a setback die on the fighters' next action. ==See also== * My revised [[Gain the Advantage]]. * A [http://community.fantasyflightgames.com/index.php?showtopic=99926 forum thread discussing this] [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] 734d37edaf529403cdde26e54c8c2b5c597a6693 2653 2652 2014-02-24T11:03:53Z Neil 2 /* Positioning, not speed */ Updated to pay more attention to ranges wikitext text/x-wiki Some house rules for [[:Category:Star Wars: Edge of the Empire|Star Wars: Edge of the Empire]] [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] I ran our first serious spaceship combat session today. The PCs were dropping off some ryll to some Geonosian buyers when four of Teemo the Hutt's cloakshape fighters dropped in. The deal was happening atop on column of a "forest" of limestone columns, similar to Zhangjiajie National Forest Park. Cue a dogfight between the crew of the Krayt Fang and a minion pair of cloakshapes, while dodging around the scenery! To help things along, I made a couple of changes while merging the combat and chase rules. ==Positioning, not speed== First of all, since everyone's dodging terrain, absolute speed isn't that important. What's important is how fast you're going for the conditions, and I use the ship's Speed rating to represent acceleration and power. At the start of every round, all pilots make a Positioning check as per pp. 240-1. This doesn't use up the pilot's Action for the round. Pilots declare their chosen speeds in turn, and then get a chance to increase (but not decrease) their speed once they've heard what their opponents are doing. After the bidding has finished, roll the Positioning check as per p. 240 and compare results with your opponent. You're after how many net successes and advantages each side gets when compared to the other. {|align=right {{Prettytable}} ! {{Hl3}}|'''Speed difference''' ! {{Hl3}}|'''Range bands moved''' |- | 0-1 || Range remains unchanged |- | 2 || Move between close and short range |- | 4 || Move between short and medium range |} Net successes add to your speed for the purposes of changing the range of the encounter. You can also spend a success to either upgrade or downgrade something in a test in the rest of the round. Advantages and disadvantages get spent as per p. 236. A Despair means you crash into something (p. 242). A Triumph means you can ignore the Navigation Hazard setback dice when rolling positioning in the next round. Once you've worked out the effective speed difference, the faster ship can change the range for the rest of the round, using costs to the right. This change also means that the Accelerate/Declerate, Fly/Drive, and Punch It manoeuvres don't make any sense. ===Example:=== The Krayt Fang is being chased by two cloakshape fighters. The Krayt's pilot calls speed 2, the fighters call speed 4. The Krayt's pilot rolls GGY (skill) + K (handling) + RR (speed & silhouette) + KK (terrain), getting two failures and an advantage. The fighters roll GGY (skill) + RRPP (speed & silhouette) + KK (terrain), getting three failures and a disadvantage. Because the fighters were faster, they can change the range from close to short range. Both sides are fairly inept, but the Krayt slightly less so. The Krayt's pilot can spend the success to effectively increase the Krayt's speed to 3, cancelling the range change. Alternatively, she can keep it to apply an upgrade or downgrade to any one test this round. In addition, the Krayt's pilot spends effectively two advantages to impose a setback die on the fighters' next action. ==See also== * My revised [[Gain the Advantage]]. * A [http://community.fantasyflightgames.com/index.php?showtopic=99926 forum thread discussing this] [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] a1a85d5afe3488061285afd41cb4112234a8fb6a 0 1634 2626 2014-02-23T19:50:12Z Neil 2 Created page with "The idea is to make it more useful, treating the action as getting a missile lock, lining up a shot, or whatever. Each net success on a Gain the Advantage check turns into an aut…" wikitext text/x-wiki The idea is to make it more useful, treating the action as getting a missile lock, lining up a shot, or whatever. Each net success on a Gain the Advantage check turns into an automatic Advantage added to the result of a subsequent gunnery check. This allows pilots to guarantee criticals, or activate a missile's Guided quality. Also, the difficulty is based on the opponent's Pilot skill + ship handling. ==Example:== Two cloakshape fighters try to Gain the Advantage on the Krayt Fang. They'd roll GGY (skill) + PPR (Krayt's pilot's skill) + B (Krayt's bad handling), getting three success, two advantages, and a despair. That'll give them an automatic three additional success to their gunnery check next round (enough for the missile's guidance systems to kick in), an immediate Evasive Manoeuvres, and a collision with the scenery. ==See also== See me revised [[Dogfighting in terrain]]. [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] e23426e3a438c3f74f83f84cecf861a9fdedc867 2628 2626 2014-02-23T19:51:50Z Neil 2 wikitext text/x-wiki The idea is to make it more useful, treating the action as getting a missile lock, lining up a shot, or whatever. Each net success on a Gain the Advantage check turns into an automatic Advantage added to the result of a subsequent gunnery check. This allows pilots to guarantee criticals, or activate a missile's Guided quality. Also, the difficulty is based on the opponent's Pilot skill + ship handling. ==Example:== Two cloakshape fighters try to Gain the Advantage on the Krayt Fang. They'd roll GGY (skill) + PPR (Krayt's pilot's skill) + B (Krayt's bad handling), getting three success, two advantages, and a despair. That'll give them an automatic three additional success to their gunnery check next round (enough for the missile's guidance systems to kick in), an immediate Evasive Manoeuvres, and a collision with the scenery. ==See also== See my revised [[Dogfighting in terrain]]. [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] a7226a04860ddd11123e0d5c60cae1de7b141bb9 2649 2628 2014-02-24T10:40:58Z Neil 2 moved [[Gaining the Advantage]] to [[Gain the Advantage]] wikitext text/x-wiki The idea is to make it more useful, treating the action as getting a missile lock, lining up a shot, or whatever. Each net success on a Gain the Advantage check turns into an automatic Advantage added to the result of a subsequent gunnery check. This allows pilots to guarantee criticals, or activate a missile's Guided quality. Also, the difficulty is based on the opponent's Pilot skill + ship handling. ==Example:== Two cloakshape fighters try to Gain the Advantage on the Krayt Fang. They'd roll GGY (skill) + PPR (Krayt's pilot's skill) + B (Krayt's bad handling), getting three success, two advantages, and a despair. That'll give them an automatic three additional success to their gunnery check next round (enough for the missile's guidance systems to kick in), an immediate Evasive Manoeuvres, and a collision with the scenery. ==See also== See my revised [[Dogfighting in terrain]]. [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] a7226a04860ddd11123e0d5c60cae1de7b141bb9 2654 2649 2014-02-24T11:05:54Z Neil 2 wikitext text/x-wiki The idea is to make it more useful, treating the action as getting a missile lock, lining up a shot, or whatever. Each net success on a Gain the Advantage check turns into an automatic Advantage added to the result of a subsequent gunnery check. This allows pilots to guarantee criticals, or activate a missile's Guided quality. The difficulty is based on the opponent's Pilot skill + ship handling. ==Example:== Two cloakshape fighters try to Gain the Advantage on the Krayt Fang. They'd roll GGY (skill) + PPR (Krayt's pilot's skill) + B (Krayt's bad handling), getting three success, two advantages, and a despair. That'll give them an automatic three additional success to their gunnery check next round (enough for the missile's guidance systems to kick in), an immediate Evasive Manoeuvres, and a collision with the scenery. ==See also== See my revised [[Dogfighting in terrain]]. [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] 363477fc8ddef8d903eb1ce2d37ba81c30e843ea 14 1635 2629 2014-02-23T19:56:04Z Neil 2 Created page with "Characters, writeups, and house rules for [http://www.fantasyflightgames.com/edge_minisite.asp?eidm=232 Star Wars: Edge of the Empire]. [[Category:RPGs]]" wikitext text/x-wiki Characters, writeups, and house rules for [http://www.fantasyflightgames.com/edge_minisite.asp?eidm=232 Star Wars: Edge of the Empire]. [[Category:RPGs]] 22711357a8abbbb796e001fd83de1b09ef53b063 14 1636 2630 2014-02-23T19:56:17Z Neil 2 Created page with "House rules for various RPGs [[Category:RPGs]]" wikitext text/x-wiki House rules for various RPGs [[Category:RPGs]] d01d2dcf5c0f3b6d86ac641498f62a0aa23af78a 0 1637 2631 2014-02-23T20:07:08Z Neil 2 Created page with "The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't…" wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. [[Category:Star Wars: Edge of the Empire]] be465284d091d79ff9a3da66ab73754de51d811f 2632 2631 2014-02-23T21:09:22Z Neil 2 wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'bann's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) [[Category:Star Wars: Edge of the Empire]] 740cc3923b9976ae34cc54189ce4639e39b4e240 2633 2632 2014-02-23T21:44:40Z Neil 2 wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships. At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. [[Category:Star Wars: Edge of the Empire]] 21d0e4e69f785cefb55a91433832a795bfa866ce 2634 2633 2014-02-23T22:13:22Z Neil 2 /* Act 2: Geonosian negotiations */ wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships. At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. [[Category:Star Wars: Edge of the Empire]] d3563ce8cca3dabcc1798ef6a93d5d610b460722 2635 2634 2014-02-23T22:28:38Z Neil 2 wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships. At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] 8134a730c490702567246115b283141b29c521d2 2636 2635 2014-02-23T22:31:05Z Neil 2 wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point, a "forest" of limestone columns like the Zhangjiajie National Forest Park. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships. At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] be1d03b5c78a2e60e09d1d98b9a64e2d90db6508 2638 2636 2014-02-23T23:00:15Z Neil 2 /* Act 2: Geonosian negotiations */ wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point, a "forest" of limestone columns like the Zhangjiajie National Forest Park. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships (half will attack the PCs as a minion group). At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] 722a3ac1bbf792c513c7df2eab1e2d512a4be3a0 2639 2638 2014-02-23T23:13:17Z Neil 2 /* Act 2: Geonosian negotiations */ wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point, a "forest" of limestone columns like the Zhangjiajie National Forest Park. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships (half will attack the PCs as a minion group). At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. {|{{Prettytable}} ! {{Hl3}} colspan=3|'''Cloakshape fighter with minion pilot (2 fighters)''' |- |''Ship'' * Silhouette: 3 * Speed: 4 * Handling: 0 : * Shields: 0 * Armour: 3 * HT: 10 + 10 * Stress: - | ''Pilot'' * Agl: 3 (Pilot, Gunnery) YGG * Int: 2 (Mechanics) YG * Cun: 2 (Perception) YG * Wil: 2 (Vigilance) YG * Pre: 2 (Cool) YG Convert Y → G when one ship eliminated. | ''Weapons'' * Light laser. Damage 5, Critical 3, Linked 1 * Concussion missile. Damage 6, Critical 3, :Breach 4, Blast 4, Guided 3, Limited Ammo 6, :Slow firing 1 |} After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] 466c1b8908b5efc87517b0ea3f27784bc3680f9e 2640 2639 2014-02-23T23:15:24Z Neil 2 /* Act 2: Geonosian negotiations */ wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point, a "forest" of limestone columns like the Zhangjiajie National Forest Park. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships (half will attack the PCs as a minion group). At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. {|{{Prettytable}} ! {{Hl3}} colspan=3|'''Cloakshape fighter with minion pilot (2 fighters)''' |- |''Ship'' * Silhouette: 3 * Speed: 4 * Handling: 0 : * Shields: 0 * Armour: 3 * HT: 10 + 10 * Stress: - | ''Pilot'' * Agl: 3 (Pilot, Gunnery) YGG * Int: 2 (Mechanics) YG * Cun: 2 (Perception) YG * Wil: 2 (Vigilance) YG * Pre: 2 (Cool) YG Convert G → Y if three ships. Convert Y → G when one ship eliminated. | ''Weapons'' * Light laser. Damage 5, Critical 3, Linked 1 * Concussion missile. Damage 6, Critical 3, :Breach 4, Blast 4, Guided 3, Limited Ammo 6, :Slow firing 1 |} After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] 0628d490e499293a22fe3938facba4686c62b429 2648 2640 2014-02-24T10:40:39Z Neil 2 /* Act 2: Geonosian negotiations */ wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point, a "forest" of limestone columns like the Zhangjiajie National Forest Park. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships (half will attack the PCs as a minion group). At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. {|{{Prettytable}} ! {{Hl3}} colspan=3|'''Cloakshape fighter with minion pilot (2 fighters)''' |- |''Ship'' * Silhouette: 3 * Speed: 4 * Handling: 0 : * Shields: 0 * Armour: 3 * HT: 10 + 10 * Stress: - | ''Pilot'' * Agl: 3 (Pilot, Gunnery) YGG * Int: 2 (Mechanics) YG * Cun: 2 (Perception) YG * Wil: 2 (Vigilance) YG * Pre: 2 (Cool) YG Convert G → Y if three ships. Convert Y → G when one ship eliminated. | ''Weapons'' * Close range: Light laser. :Damage 5, Critical 3, Linked 1 * Short range: Concussion missile. :Damage 6, Critical 3, Breach 4, :Blast 4, Guided 3, Limited Ammo 6, :Slow firing 1 |} You may want to use the [[Dogfighting in terrain]] and alternative [[Gain the Advantage]] rules. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] 07cb275745d1b0eb0c2bffac87cf85349f90598a 0 1638 2641 2014-02-24T08:59:16Z Neil 2 Created page with "* Species: '''Rodian''' * Career: '''Bounty Hunter''' (Survivalist) ==Characteristics== * Brawn: 3 * Agility: 3 * Intellect: 2 * Cunning: 2 * Willpower: 1 * Presence: 3 ===Deri…" wikitext text/x-wiki * Species: '''Rodian''' * Career: '''Bounty Hunter''' (Survivalist) ==Characteristics== * Brawn: 3 * Agility: 3 * Intellect: 2 * Cunning: 2 * Willpower: 1 * Presence: 3 ===Derived characteristics=== * Soak 3 * Wound threshold: 13 * Strain threshold: 11 ==Skills== * Cool: 1 * Deception: 1 * Medicine: 1 * Pilot (planetary)*: 1 * Pilot (space)*: 1 * Resilience*: 1 * Stealth: 1 * Streetwise: 1 * Survival: 1 * Vigilance: 2 * Ranged (light): 1 * Ranged (heavy): 2 * Xenology: 1 :* Career skill. ==Talents== * Expert Tracker 3 * Forager * Soft Spot * Stalker 2 * Grit * Hunter * Swift [[Category:Star Wars: Edge of the Empire]] 86260574b5d0b7a8c421771cd96e447a40f7ebde 2642 2641 2014-02-24T09:00:31Z Neil 2 /* Skills */ wikitext text/x-wiki * Species: '''Rodian''' * Career: '''Bounty Hunter''' (Survivalist) ==Characteristics== * Brawn: 3 * Agility: 3 * Intellect: 2 * Cunning: 2 * Willpower: 1 * Presence: 3 ===Derived characteristics=== * Soak 3 * Wound threshold: 13 * Strain threshold: 11 ==Skills== * Cool: 1 * Deception: 1 * Medicine: 1 * Pilot (planetary)*: 1 * Pilot (space)*: 1 * Resilience*: 1 * Stealth: 1 * Streetwise*: 1 * Survival*: 1 * Vigilance*: 2 * Ranged (light): 1 * Ranged (heavy)*: 2 * Xenology*: 1 :* Career skill. ==Talents== * Expert Tracker 3 * Forager * Soft Spot * Stalker 2 * Grit * Hunter * Swift [[Category:Star Wars: Edge of the Empire]] c946e5c7ff8a0081b85e7eff6eb4abc9b396b187 2645 2642 2014-02-24T09:37:57Z Neil 2 /* Skills */ wikitext text/x-wiki * Species: '''Rodian''' * Career: '''Bounty Hunter''' (Survivalist) ==Characteristics== * Brawn: 3 * Agility: 3 * Intellect: 2 * Cunning: 2 * Willpower: 1 * Presence: 3 ===Derived characteristics=== * Soak 3 * Wound threshold: 13 * Strain threshold: 11 ==Skills== {| |rowspan=2| * Cool: 1 * Deception: 1 * Medicine: 1 * Pilot (planetary)*: 1 * Pilot (space)*: 1 * Resilience*: 1 * Stealth: 1 * Streetwise*: 1 * Survival*: 1 * Vigilance*: 2 |valign=top| * Ranged (light): 1 * Ranged (heavy)*: 2 |- |valign=top| * Knowledge (Xenology)*: 1 |} :<nowiki>*</nowiki> Career skill. ==Talents== * Expert Tracker 3 * Forager * Soft Spot * Stalker 2 * Grit * Hunter * Swift [[Category:Star Wars: Edge of the Empire]] 5ab80bde137493037c3147bf69d45c88e72b9a36 2647 2645 2014-02-24T09:55:30Z Neil 2 /* Characteristics */ wikitext text/x-wiki * Species: '''Rodian''' * Career: '''Bounty Hunter''' (Survivalist) ==Characteristics== {| !align=left|'''Basic''' !align=left|'''Derived''' |-valign=top | * Brawn: 3 * Agility: 3 * Intellect: 2 * Cunning: 2 * Willpower: 1 * Presence: 3 | * Soak 3 * Wound threshold: 13 * Strain threshold: 11 |} ==Skills== {| |rowspan=2| * Cool: 1 * Deception: 1 * Medicine: 1 * Pilot (planetary)*: 1 * Pilot (space)*: 1 * Resilience*: 1 * Stealth: 1 * Streetwise*: 1 * Survival*: 1 * Vigilance*: 2 |valign=top| * Ranged (light): 1 * Ranged (heavy)*: 2 |- |valign=top| * Knowledge (Xenology)*: 1 |} :<nowiki>*</nowiki> Career skill. ==Talents== * Expert Tracker 3 * Forager * Soft Spot * Stalker 2 * Grit * Hunter * Swift [[Category:Star Wars: Edge of the Empire]] 64a31f73a6d66f4a000b7f59dbb364e096df5a1e 0 1639 2643 2014-02-24T09:33:20Z Neil 2 Created page with "* Species: '''Droid''' * Career: '''Technician''' (Mechanic) ==Characteristics== * Brawn: 3 * Agility: 2 * Intellect: 3 * Cunning: 2 * Willpower: 2 * Presence: 2 ===Derived cha…" wikitext text/x-wiki * Species: '''Droid''' * Career: '''Technician''' (Mechanic) ==Characteristics== * Brawn: 3 * Agility: 2 * Intellect: 3 * Cunning: 2 * Willpower: 2 * Presence: 2 ===Derived characteristics=== * Soak 5 * Wound threshold: 15 * Strain threshold: 12 ==Skills== * Astrogation*: 1 * Coercion: 1 * Computers*: 1 * Mechanics*: 4 * Medicine: 1 * Perception*: 1 * Pilot (planetary)*: 1 * Pilot (space)*: 1 * Skulduggery*: 1 * Brawl*: 1 * Knowledge (Outer Rim)*: 1 :* Career skill. ==Talents== * Toughened * Solid Repairs 2 * Redundant Systems * Enduring 2 * Contraption * Bad Motivator [[Category:Star Wars: Edge of the Empire]] 7dd7574ac66e33e624694646504e75179f39f595 2644 2643 2014-02-24T09:37:53Z Neil 2 /* Skills */ wikitext text/x-wiki * Species: '''Droid''' * Career: '''Technician''' (Mechanic) ==Characteristics== * Brawn: 3 * Agility: 2 * Intellect: 3 * Cunning: 2 * Willpower: 2 * Presence: 2 ===Derived characteristics=== * Soak 5 * Wound threshold: 15 * Strain threshold: 12 ==Skills== {| |rowspan=2| * Astrogation*: 1 * Coercion: 1 * Computers*: 1 * Mechanics*: 4 * Medicine: 1 * Perception*: 1 * Pilot (planetary)*: 1 * Pilot (space)*: 1 * Skulduggery*: 1 |valign=top| * Brawl*: 1 |- |valign=top| * Knowledge (Outer Rim)*: 1 |} :<nowiki>*</nowiki> Career skill. ==Talents== * Toughened * Solid Repairs 2 * Redundant Systems * Enduring 2 * Contraption * Bad Motivator [[Category:Star Wars: Edge of the Empire]] d08f72242f11b753ff7351ffeb6c665e7b45a9bc 2646 2644 2014-02-24T09:55:27Z Neil 2 /* Characteristics */ wikitext text/x-wiki * Species: '''Droid''' * Career: '''Technician''' (Mechanic) ==Characteristics== {| !align=left|'''Basic''' !align=left|'''Derived''' |-valign=top | * Brawn: 3 * Agility: 2 * Intellect: 3 * Cunning: 2 * Willpower: 2 * Presence: 2 | * Soak 5 * Wound threshold: 15 * Strain threshold: 12 |} ==Skills== {| |rowspan=2| * Astrogation*: 1 * Coercion: 1 * Computers*: 1 * Mechanics*: 4 * Medicine: 1 * Perception*: 1 * Pilot (planetary)*: 1 * Pilot (space)*: 1 * Skulduggery*: 1 |valign=top| * Brawl*: 1 |- |valign=top| * Knowledge (Outer Rim)*: 1 |} :<nowiki>*</nowiki> Career skill. ==Talents== * Toughened * Solid Repairs 2 * Redundant Systems * Enduring 2 * Contraption * Bad Motivator [[Category:Star Wars: Edge of the Empire]] eed9ca3ca229db0636d6dadd019e52971a9bcd97 0 1633 2655 2653 2014-02-24T11:25:17Z Neil 2 /* Example: */ wikitext text/x-wiki Some house rules for [[:Category:Star Wars: Edge of the Empire|Star Wars: Edge of the Empire]] [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] I ran our first serious spaceship combat session today. The PCs were dropping off some ryll to some Geonosian buyers when four of Teemo the Hutt's cloakshape fighters dropped in. The deal was happening atop on column of a "forest" of limestone columns, similar to Zhangjiajie National Forest Park. Cue a dogfight between the crew of the Krayt Fang and a minion pair of cloakshapes, while dodging around the scenery! To help things along, I made a couple of changes while merging the combat and chase rules. ==Positioning, not speed== First of all, since everyone's dodging terrain, absolute speed isn't that important. What's important is how fast you're going for the conditions, and I use the ship's Speed rating to represent acceleration and power. At the start of every round, all pilots make a Positioning check as per pp. 240-1. This doesn't use up the pilot's Action for the round. Pilots declare their chosen speeds in turn, and then get a chance to increase (but not decrease) their speed once they've heard what their opponents are doing. After the bidding has finished, roll the Positioning check as per p. 240 and compare results with your opponent. You're after how many net successes and advantages each side gets when compared to the other. {|align=right {{Prettytable}} ! {{Hl3}}|'''Speed difference''' ! {{Hl3}}|'''Range bands moved''' |- | 0-1 || Range remains unchanged |- | 2 || Move between close and short range |- | 4 || Move between short and medium range |} Net successes add to your speed for the purposes of changing the range of the encounter. You can also spend a success to either upgrade or downgrade something in a test in the rest of the round. Advantages and disadvantages get spent as per p. 236. A Despair means you crash into something (p. 242). A Triumph means you can ignore the Navigation Hazard setback dice when rolling positioning in the next round. Once you've worked out the effective speed difference, the faster ship can change the range for the rest of the round, using costs to the right. This change also means that the Accelerate/Declerate, Fly/Drive, and Punch It manoeuvres don't make any sense. ===Example:=== The Krayt Fang is being chased by two cloakshape fighters. The Krayt's pilot calls speed 2, the fighters call speed 4. The Krayt's pilot rolls GGY (skill) + K (handling) + RR (speed & silhouette) + KK (terrain), getting two failures and an advantage. The fighters roll GGY (skill) + RRPP (speed & silhouette) + KK (terrain), getting three failures and a disadvantage. Because the fighters were faster, they can change the range from short to close range. Both sides are fairly inept, but the Krayt slightly less so. The Krayt's pilot can spend the success to effectively increase the Krayt's speed to 3, cancelling the range change. Alternatively, she can keep it to apply an upgrade or downgrade to any one test this round. In addition, the Krayt's pilot spends effectively two advantages to impose a setback die on the fighters' next action. ==See also== * My revised [[Gain the Advantage]]. * A [http://community.fantasyflightgames.com/index.php?showtopic=99926 forum thread discussing this] [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] 0a364ccbdc5e6e59c5cb9254507828d57715221a 2656 2655 2014-02-24T19:33:38Z Neil 2 /* Positioning, not speed */ Changed bidding to be for additional setback dice. wikitext text/x-wiki Some house rules for [[:Category:Star Wars: Edge of the Empire|Star Wars: Edge of the Empire]] [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] I ran our first serious spaceship combat session today. The PCs were dropping off some ryll to some Geonosian buyers when four of Teemo the Hutt's cloakshape fighters dropped in. The deal was happening atop on column of a "forest" of limestone columns, similar to Zhangjiajie National Forest Park. Cue a dogfight between the crew of the Krayt Fang and a minion pair of cloakshapes, while dodging around the scenery! To help things along, I made a couple of changes while merging the combat and chase rules. ==Positioning, not speed== First of all, since everyone's dodging terrain, absolute speed isn't that important. What's important is how fast you're going for the conditions, and I use the ship's Speed rating to represent acceleration and power. At the start of every round, all pilots make a Positioning check as per pp. 240-1. This doesn't use up the pilot's Action for the round. Pilots independently declare their chosen speeds for the round. They can then bid, in turn, to increase the setback dice from the terrain, reflecting taking additional risks. All pilots must take additional setback dice equal to the highest bid. After the bidding has finished, roll the Positioning check as per p. 240 and compare results with your opponent. You're after how many net successes and advantages each side gets when compared to the other. {|align=right {{Prettytable}} ! {{Hl3}}|'''Speed difference''' ! {{Hl3}}|'''Range bands moved''' |- | 0-1 || Range remains unchanged |- | 2 || Move between close and short range |- | 4 || Move between short and medium range |} Net successes add to your speed for the purposes of changing the range of the encounter. You can also spend a success to either upgrade or downgrade something in a test in the rest of the round. Advantages and disadvantages get spent as per p. 236. A Despair means you crash into something (p. 242). A Triumph means you can ignore the Navigation Hazard setback dice when rolling positioning in the next round. Once you've worked out the effective speed difference, the faster ship can change the range for the rest of the round, using costs to the right. This change also means that the Accelerate/Declerate, Fly/Drive, and Punch It manoeuvres don't make any sense. ===Example:=== The Krayt Fang is being chased by two cloakshape fighters. The Krayt's pilot calls speed 2, the fighters call speed 4. The Krayt's pilot rolls GGY (skill) + K (handling) + RR (speed & silhouette) + KK (terrain), getting two failures and an advantage. The fighters roll GGY (skill) + RRPP (speed & silhouette) + KK (terrain), getting three failures and a disadvantage. Because the fighters were faster, they can change the range from short to close range. Both sides are fairly inept, but the Krayt slightly less so. The Krayt's pilot can spend the success to effectively increase the Krayt's speed to 3, cancelling the range change. Alternatively, she can keep it to apply an upgrade or downgrade to any one test this round. In addition, the Krayt's pilot spends effectively two advantages to impose a setback die on the fighters' next action. ==See also== * My revised [[Gain the Advantage]]. * A [http://community.fantasyflightgames.com/index.php?showtopic=99926 forum thread discussing this] [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] 16a518e1dec27576ec36cf2dba63efe8b9d69eb5 0 1637 2657 2648 2014-03-01T12:47:02Z Neil 2 wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point, a "forest" of limestone columns like the Zhangjiajie National Forest Park. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships (half will attack the PCs as a minion group). At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. {|{{Prettytable}} ! {{Hl3}} colspan=3|'''Cloakshape fighter with minion pilot (2 fighters)''' |- |''Ship'' * Silhouette: 3 * Speed: 4 * Handling: 0 : * Shields: 0 * Armour: 3 * HT: 10 + 10 * Stress: - | ''Pilot'' * Agl: 3 (Pilot, Gunnery) YGG * Int: 2 (Mechanics) YG * Cun: 2 (Perception) YG * Wil: 2 (Vigilance) YG * Pre: 2 (Cool) YG Convert G → Y if three ships. Convert Y → G when one ship eliminated. | ''Weapons'' * Close range: Light laser. :Damage 5, Critical 3, Linked 1 * Short range: Concussion missile. :Damage 6, Critical 3, Breach 4, :Blast 4, Guided 3, Limited Ammo 6, :Slow firing 1 |} You may want to use the [[Dogfighting in terrain]] and alternative [[Gain the Advantage]] rules. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. ==Act 3: Return to Mos Shuuta== This act I created entirely afresh. We know that Teemo the Hutt has been engaged in the ryll trade to finance his assembly of a battle droid force. He plans to use the droids to overthrow Jabba. That means that he wants to keep both of these activities secret, so they can't take place in Mos Shuuta. Teemo has recently bought a moisture farm called Doni Skanthi about thirty kilometres from Mos Shuuta. He's using this as the base of his new operations. There are a number of clues that point to Doni Skanthi. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] fa6e9eb645d89d3bc52b0144eda86cad7e688b4f 2658 2657 2014-03-01T18:31:56Z Neil 2 /* Act 3: Return to Mos Shuuta */ wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point, a "forest" of limestone columns like the Zhangjiajie National Forest Park. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships (half will attack the PCs as a minion group). At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. {|{{Prettytable}} ! {{Hl3}} colspan=3|'''Cloakshape fighter with minion pilot (2 fighters)''' |- |''Ship'' * Silhouette: 3 * Speed: 4 * Handling: 0 : * Shields: 0 * Armour: 3 * HT: 10 + 10 * Stress: - | ''Pilot'' * Agl: 3 (Pilot, Gunnery) YGG * Int: 2 (Mechanics) YG * Cun: 2 (Perception) YG * Wil: 2 (Vigilance) YG * Pre: 2 (Cool) YG Convert G → Y if three ships. Convert Y → G when one ship eliminated. | ''Weapons'' * Close range: Light laser. :Damage 5, Critical 3, Linked 1 * Short range: Concussion missile. :Damage 6, Critical 3, Breach 4, :Blast 4, Guided 3, Limited Ammo 6, :Slow firing 1 |} You may want to use the [[Dogfighting in terrain]] and alternative [[Gain the Advantage]] rules. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. ==Act 3: Return to Mos Shuuta== This act I created entirely afresh. We know that Teemo the Hutt has been engaged in the ryll trade to finance his assembly of a battle droid force. He plans to use the droids to overthrow Jabba. That means that he wants to keep both of these activities secret, so they can't take place in Mos Shuuta. Teemo has recently bought a moisture farm called Doni Skanthi about thirty kilometres from Mos Shuuta. He's using this as the base of his new operations. There are a number of clues that point to Doni Skanthi. * Teemo has been recently spending a lot of time away from his compound and the rumours point to a moisture farm he's just bought. * Teemo commissioned a lot of building work at the farm, mainly digging out and enlarging the underground storage spaces. He hasn't paid the workers all they were owed. The workers, led by Orron Laga (human male), are protesting both directly at Teemo's compound and through the Imperial authorities. They're not having much luck with either but will happily tell their woes to anyone who buys them a drink. The workers don't know what the expanded chambers were for, but the farm itself seemed run down. * Teemo's hired a couple of extra droid technicians recently (Koobis Kena, Rodian female and Do'sara, Twi'lek male). Two days ago, he started a seemingly desperate search for another couple of technicians and is buying up as many humanoid-form droids as he can. Soont Izes (human male) has just been hired and is about to leave for Doni Skanthi. The original two technicians haven't been seen for the last few days, but when they were in Mos Shuuta they chatted about the droid repair work they were doing at Doni Skanthi. * If any of the cloakshape fighters survived the earlier dogfight, and the PCs can get access to the air traffic control records, they can learn the some damaged fighters have landed at Doni Skanthi. ===Doni Skanthi=== If the PCs visit Doni Skanthi, they see a fairly typical moisture farm. Most of the farm "buildings" are underground, with entrances into a central well. There's also a large above-ground fenced enclosure containing a few vehicles and outhouses. At the rear of the farm is a large pile of earth and rubble, spoils from Laga's excavations. Anyone looking at the pile will be able to tell it's been there for few days or weeks. Outside the compound is the Dangon Travelling Circus, led by Kreiva Dangon (human female). The circus has a couple of beat-up speeder trucks, a few people on dewbacks, and a rancor tethered to one of the trucks. Kreiva wants to put on a quick show for the "farmers" in return for some shelter from the sun and water for her animals. The people inside the farm aren't having any of it, and things are beginning to get a bit heated. The farm is occupied by three of Teemo's Gamorrean thugs, two Imperial Stormtroopers in civilian clothes, and Koobis Kena and Do'sara the technicians. They won't let anyone into the farm. The thugs are all brawn and bluster, but the stormtroopers are cool professionals. The technicians will generally stay indoors, working on the droids, but will come out if something interesting looks to be happening. If the PCs get into the farm, they find precious little in the way of active farming going on. Two chambers have obviously been worked on recently. One contains several hover-pallets of ryll. The other is the droid storage area. It contains about thirty B1 battle droids, without heads and inactive. A couple have heads spliced on from other droids. The computer in here is trivially slicable to find a set of plans for killing Jabba, ranging from a surgical assassination on his sail barge to a full-frontal assault on his compound. Plans for Teemo's overthrow of Jabba can also be found in Teemo's compound, as described in the adventure. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] 590bf0f5ca6e0d46af812f015020b298b749ef1c 2659 2658 2014-03-01T18:35:25Z Neil 2 /* Act 3: Return to Mos Shuuta */ wikitext text/x-wiki The [http://www.fantasyflightgames.com/edge_minisite_sec.asp?eidm=224&esem=4 Long Arm of the Hutt] adventure is a good one, but I've made a few changes for bits I thought weren't wonderful. ==Act 1: Rugged road to the ryll mines== I played this one fairly straight. The only change I made was to name a few NPCs in the settlement of New Meen and among Drombb's thugs. '''Bura'bann''' the PCs already know, having rescued him from Trex's holding cell. In his absence, New Meen is being run by his husband, '''Gunn'bann''', another aged Twi'lek. Their son, '''Shom'synda''' (genesplicing, if anyone asks), has decided that the grass is greener on the other side and has thrown his lot in with Drombb. Shom'synda's wife, '''Ayyn'synda''' is still in the village but is trying to persuade the other villagers to take Drombb's money. Many have. When the PCs arrive, they see a family putting the last boxes on a transport speeder as they pack up to leave. As they leave, Shom hangs out of the cab, waving a wad of cash around and calling on people (fairly robustly) to take Drombb's cash and leave. At the town meeting that evening, Gunn says how the situation has worsened recently and she expects Drombb and Shom to resort to direct violence soon. Ayyn says that the situation is hopeless and urges everyone to leave soon. Later that night, move into the ''Drunk and disorderly'' episode. Shom is one of the drunkards, which makes combat with the drunkards a bit more complicated as it takes place in front of Bura and Gunn. (As an aside, the PCs captured one of the bounty hunters from the lylek den, but failed to tie him up securely (failure and a threat). When he escaped when the PCs attacked Drombb's compound, Gunn was a convenient hostage for him to take. The bounty hunter's attempt to trade Gunn for the PCs didn't go too well for him.) ==Act 2: Geonosian negotiations== I made a lot of changes to this one, cutting out the second Duke and the whole of the party. When the PCs arrive back in Nabat, they're made an offer by Nyn. With Drombb now out of action, she's intercepted his latest shipment of ryll. It's destined for some Geonosians in exchange for some of their weapons. The trade will take place soon on a deserted part of Rodia. Nyn wants the weapons, but needs a light freighter to transport the spice there and the weapons back. She doesn't know exactly what the weapons are. She's arranged for the trick transponder to be reinstalled on the ''Krayt Fang'' and will happily tell Teemo the code if the PCs double cross her. Ota doesn't turn up yet. [[Image:Zhangjiajie-National-Forest-Park.jpg|right|thumb|400px|Zhangjiajie National Forest Park]] When the PCs arrive after an uneventful trip, they see another light freighter already landed at the rendesvous point, a "forest" of limestone columns like the Zhangjiajie National Forest Park. When the PCs land, the Geonosian Captain Krylla is jumpy and hostile to the PCs, saying that he hopes that this won't be another double cross like last time, and demanding to see the spice before he hands over any of the heads. Just as the PCs are getting to the bottom of this mix-up, but before any goods can be exchanged, four or six Teemo's Cloakshape fighters appear and attack both ships (half will attack the PCs as a minion group). At first, the fighter pilots are friendly to the PCs and hail Trex on the ''Krayt Fang'', but soon change their tune when they work it out. {|{{Prettytable}} ! {{Hl3}} colspan=3|'''Cloakshape fighter with minion pilot (2 fighters)''' |- |''Ship'' * Silhouette: 3 * Speed: 4 * Handling: 0 : * Shields: 0 * Armour: 3 * HT: 10 + 10 * Stress: - | ''Pilot'' * Agl: 3 (Pilot, Gunnery) YGG * Int: 2 (Mechanics) YG * Cun: 2 (Perception) YG * Wil: 2 (Vigilance) YG * Pre: 2 (Cool) YG Convert G → Y if three ships. Convert Y → G when one ship eliminated. | ''Weapons'' * Close range: Light laser. :Damage 5, Critical 3, Linked 1 * Short range: Concussion missile. :Damage 6, Critical 3, Breach 4, :Blast 4, Guided 3, Limited Ammo 6, :Slow firing 1 |} You may want to use the [[Dogfighting in terrain]] and alternative [[Gain the Advantage]] rules. After any battle, survivors have to escape quickly before Rodian system defence forces turn up to arrest everyone they can find. if the PCs want the weapons, they'll probably take up Krylla's offer to return to Geonosis. In audience with Duke Dimmock, the story comes out. Teemo was buying a large force of battle droids from Dimmock, paying for them with ryll. Someone had attacked the previous shipment and stolen the droid parts. The PCs turned up in the middle of the final delivery, the heads (control units that make the droids operational). Dimmock is now convinced that Teemo is double-crossing him and what the PCs tell him about the fighter pilots only corroborates this. The PCs may negotiate the purchase of more suitable weapons for the Nabat militants for the ryll they have. Ota's in the audience with Duke Dimmock. After the Duke dismisses the PCs, Ota goes with them. He introduces himself as an agent of Jabba the Hutt. If Teemo has really been assembling a battle droid force as well as getting into the ryll trade, Jabba will be very upset. Ota makes the PCs an offer they can't refuse. If the PCs can get proof of Teemo's plans to usurp Jabba, Jabba will make the PCs' problem with Teemo disappear. ==Act 3: Return to Mos Shuuta== This act I created entirely afresh. We know that Teemo the Hutt has been engaged in the ryll trade to finance his assembly of a battle droid force. He plans to use the droids to overthrow Jabba. That means that he wants to keep both of these activities secret, so they can't take place in Mos Shuuta. Teemo has recently bought a moisture farm called Doni Skanthi about thirty kilometres from Mos Shuuta. He's using this as the base of his new operations. There are a number of clues that point to Doni Skanthi. * Teemo has been recently spending a lot of time away from his compound and the rumours point to a moisture farm he's just bought. * Teemo commissioned a lot of building work at the farm, mainly digging out and enlarging the underground storage spaces. He hasn't paid the workers all they were owed. The workers, led by Orron Laga (human male), are protesting both directly at Teemo's compound and through the Imperial authorities. They're not having much luck with either but will happily tell their woes to anyone who buys them a drink. The workers don't know what the expanded chambers were for, but the farm itself seemed run down. * Teemo's hired a couple of extra droid technicians recently (Koobis Kena, Rodian female and Do'sara, Twi'lek male). Two days ago, he started a seemingly desperate search for another couple of technicians and is buying up as many humanoid-form droids as he can. Soont Izes (human male) has just been hired and is about to leave for Doni Skanthi. The original two technicians haven't been seen for the last few days, but when they were in Mos Shuuta they chatted about the droid repair work they were doing at Doni Skanthi. * If any of the cloakshape fighters survived the earlier dogfight, and the PCs can get access to the air traffic control records, they can learn the some damaged fighters have landed at Doni Skanthi. ===Doni Skanthi=== If the PCs visit Doni Skanthi, they see a fairly typical moisture farm. Most of the farm "buildings" are underground, with entrances into a central well. There's also a large above-ground fenced enclosure containing a few vehicles and outhouses. At the rear of the farm is a large pile of earth and rubble, spoils from Laga's excavations. Anyone looking at the pile will be able to tell it's been there for few days or weeks. Behind the rubble pile are any surviving cloakshape fighters. Repairs have started on them, but not got very far. Outside the compound is the Dangon Travelling Circus, led by Kreiva Dangon (human female). The circus has a couple of beat-up speeder trucks, a few people on dewbacks, and a rancor tethered to one of the trucks. Kreiva wants to put on a quick show for the "farmers" in return for some shelter from the sun and water for her animals. The people inside the farm aren't having any of it, and things are beginning to get a bit heated. The farm is occupied by three of Teemo's Gamorrean thugs, two Imperial Stormtroopers in civilian clothes, and Koobis Kena and Do'sara the technicians. They won't let anyone into the farm. The thugs are all brawn and bluster, but the stormtroopers are cool professionals. The technicians will generally stay indoors, working on the droids, but will come out if something interesting looks to be happening. If the PCs get into the farm, they find precious little in the way of active farming going on. Two chambers have obviously been worked on recently. One contains several hover-pallets of ryll. The other is the droid storage area. It contains about thirty B1 battle droids, without heads and inactive. A couple have heads spliced on from other droids. The computer in here is trivially slicable to find a set of plans for killing Jabba, ranging from a surgical assassination on his sail barge to a full-frontal assault on his compound. Plans for Teemo's overthrow of Jabba can also be found in Teemo's compound, as described in the adventure. [[Category:Star Wars: Edge of the Empire]] [[Category:Scenarios]] ff85948a5acf560ab9661aad9d5896bf8470aabf 0 1640 2660 2014-03-23T20:47:45Z Neil 2 Created page with "Some house rules I use ==Character generation== By-the-book starting characters are pretty pathetic and undifferentiated. This is mainly because everyone spends all their starti…" wikitext text/x-wiki Some house rules I use ==Character generation== By-the-book starting characters are pretty pathetic and undifferentiated. This is mainly because everyone spends all their starting XP on attribute increases, leaving very little for skills and talents. I now give all starting PCs a big chunk of additional XP to spend before play, but this additional XP can't be spent on attributes. Increases range from 50XP in a six-player game to 150XP in a two-player game. ==Character advancement== ===Talent trees=== I can't see any real reason for the limited links in the talent trees. I play it that every talent is connected to all horizontally and vertically adjacent talents. You still need to climb the ladder to get to the most expensive ones, but you can make choices. (The argument given for the limited choices is that "no self-respecting Slicer wouldn't start off buying this list of Talents." That may be true, but I think that should be a choice of the player, not the designer.) ==Skill use== Sw:EotE apportions skills somewhat differently from many other RPGs. This clarifies what skill does what. '''Perception''' and '''Vigilance''' :Perception should probably be renamed ''Investigate'' or ''Scrutinise'': it's an active, concious examination of an object or scene. Vigilance is the passive perception of your surroundings. In published adventures, many Perception checks should be replaced with Vigilance checks. '''Cool''' and '''Discipline''' :Cool is about how you present yourself to the world ("putting on a brave face"), while Discipline is about controlling your emotions. Both can be used for fear checks, but the effects of using Cool to overcome fear will typically last for a much shorter time than using Discipline. :Cool is also the skill to use when trying to read the emotions and motivations of someone else. '''Piloting (Space)''' and '''Piloting (Planetary)''' :If the movement is happening near the ground (or water, or somesuch), it's Planetary. If the movement is happening in mostly free space, it's Space. [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] 2912ccf4b9be482e524140f1fd12dfd1495ff4c3 2661 2660 2014-03-26T08:59:52Z Neil 2 Added compels wikitext text/x-wiki Some house rules I use ==Character generation== By-the-book starting characters are pretty pathetic and undifferentiated. This is mainly because everyone spends all their starting XP on attribute increases, leaving very little for skills and talents. I now give all starting PCs a big chunk of additional XP to spend before play, but this additional XP can't be spent on attributes. Increases range from 50XP in a six-player game to 150XP in a two-player game. ==Character advancement== ===Talent trees=== I can't see any real reason for the limited links in the talent trees. I play it that every talent is connected to all horizontally and vertically adjacent talents. You still need to climb the ladder to get to the most expensive ones, but you can make choices. (The argument given for the limited choices is that "no self-respecting Slicer wouldn't start off buying this list of Talents." That may be true, but I think that should be a choice of the player, not the designer.) ==Skill use== Sw:EotE apportions skills somewhat differently from many other RPGs. This clarifies what skill does what. '''Perception''' and '''Vigilance''' :Perception should probably be renamed ''Investigate'' or ''Scrutinise'': it's an active, concious examination of an object or scene. Vigilance is the passive perception of your surroundings. In published adventures, many Perception checks should be replaced with Vigilance checks. '''Cool''' and '''Discipline''' :Cool is about how you present yourself to the world ("putting on a brave face"), while Discipline is about controlling your emotions. Both can be used for fear checks, but the effects of using Cool to overcome fear will typically last for a much shorter time than using Discipline. :Cool is also the skill to use when trying to read the emotions and motivations of someone else. '''Piloting (Space)''' and '''Piloting (Planetary)''' :If the movement is happening near the ground (or water, or somesuch), it's Planetary. If the movement is happening in mostly free space, it's Space. ==Obligation and Motivation as Aspects== Some of the Obligations (such as Bounty or Favour) are easy to work into an ongoing game. Others (such as Oath and Duty) and all the Motivations are difficult for the GM to incorporate into play. (I'm not a fan of the stress reduction mechanic, and judging people when passing out XP is too divisive.) Instead, I treat Obligations and Motivations similarly to Aspects in Fate, in that they can be Compelled by the GM or players. The Compel takes the form of one of: * You have ____ obligation/motivation and are in ____ situation, so it makes sense that, unfortunately, ____ would happen to you. Damn your luck. * You have ____ obligation/motivation in ____ situation, so it makes sense that you’d decide to ____. This goes wrong when ____ happens. For instance, * "You have the ''Dutybound'' obligation and ''are trying to get help from that organisation'', so it makes sense that, unfortunately, ''the official you're dealing with questions your loyalty.'' Damn your luck." * "You have the '''droid rights'' motivation while ''a droid is being pushed around by the bar partrons,'' so it makes sense that you'd decide to ''stop them.'' This goes wrong when ''they start preparing for a fight.''" If the player accepts the Compel, flip a Destiny token from Dark to Light. If the player doesn't want to be subject to the Compel, they can flip a Destiny token from Light to Dark and the proposed event doesn't happen. [[Category:Star Wars: Edge of the Empire]] [[Category: House rules]] b158eb2ad5d3400d9f646ce53c3c3f1dea4e4aaa 0 1481 2662 1679 2014-04-09T08:44:40Z Neil 2 wikitext text/x-wiki I use a few house rules for the Synergy system, as used in [[:Category:Blue Planet|Blue Planet]]. They are all minor, and have very little effect on the already lightweight Synergy game system. See also the [http://bpwiki.beachheadonline.com/index.php5?title=Errata#Version_2 errata] on the [http://bpwiki.beachheadonline.com/index.php5?title=Main_Page Blue Planet wiki]. == Character Generation == I made a few restrictions on the character generation choices available to the players. * Characters are at the Exceptional power level, apart from having Everyday attributtes (i.e. sum to zero) * There are no systemic osmoform aquaforms (squids): gills on a mammal would cause virtually instant and terminal hypothermia. Diving reflex aquaforms, however, are perfectly believable. * Hybrids look far more like normal humans than how they are presented in the rules: genetic modification to increase muscle mass, say, will have no effect on the development of, say, the nose. Silvas tend to look like people with acromegaly, with large jaws and hands and increased muscle mass. Cats are thin, nervous, and constantly jumpy. == Open-ended dice rolls == When making a task attempt, natural rolls of '1' and '10' are open ended. If the lowest die rolled is a '1', add 10 to the Target Number and roll again (but only reroll the dice that show a '1'). If the lowest die rolled is a '10', subtract 10 from the Target number and roll again. Multiple consecutive rolls of '1' or '10' keep moving the Target Number. This mechanism allows attempts at tasks that, through various factors, have negative target numbers. :(Inspired by Mike Z's house rule) == Augmentation == * When two or more skills are applicable to a certain task (e.g. Pilot and Computer when trying to override a sabotaged autopilot), the secondary skill can give a bonus of one-third its value (round down) to the main skill value. :(Inspired by [[:Category:HeroQuest|HeroQuest]]) == Close Combat == First is the single erratum from the published rulebooks: * Unarmed comabat strikes have a damage rating of Strength + 1 Second is the house rule. * When making an offensive action (Strike or Throw), but before dice are rolled, an attacker can nominate to forgo an attempt to cause damage, but instead acquire a combat advantage. In this case, the attacker's net Action Value (attacker's AV - defender's AV) is applied as a bonus to the attacker's next action. Note that the net AV can be negative, giving the attacker a penalty. Also note that the combat advantage resets to zero if it is not used in the attacker's next action. == Equipment == * The electronic weapon scope does not give a bonus to all shots; instead, it alleviates any range penalites by 2. Its range-trebling effect remains unchanged. * The targeting interlink does not interface with programmed reflexes to provide a 'never miss' effect. It still gives a +2/+3 bonus to all shots when used. == Biomods == * The diver aquaform includes the improved blood oxygenation modification. Characters with either modification have the union of the stated benefits: they can hold their breath for up to 10 minutes while performing strenuous activity, and up to an hour if resting. * Multiglands can produce a number of effects. See [[what multiglands can do for you]] for details. [[Category:Blue Planet]] [[Category:House rules]] d2b39b7a6d29ef9913285252f79ef0e84588ac03 0 1530 2663 2435 2014-04-11T15:23:46Z Neil 2 /* Initial configuration */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <code>/etc/ssl/domain</code>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create a list of key revocations root@server:~# echo '01' > domain/crlnumber * Create the <code>/etc/ssl/openssl.cnf</code> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME = . RANDFILE = $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids # To use this configuration file with the "-extfile" option of the # "openssl x509" utility, name here the section containing the # X.509v3 extensions to use: # extensions = # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca' and 'req'. # Add a simple OID like this: # testoid1=1.2.3.4 # Or use config file substitution like this: # testoid2=${testoid1}.5.6 #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. #unique_subject = no # Set to 'no' to allow creation of # several ctificates with same subject. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/crl.pem # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert # Comment out the following two lines for the "traditional" # (and highly broken) format. name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options # Extension copying option: use with caution. # copy_extensions = copy # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs # so this is commented out by default to leave a V1 CRL. # crlnumber must also be commented out to leave a V1 CRL. # crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering # A few difference way of specifying how similar the request should look # For type CA, the listed attributes must be the same, and the optional # and supplied fields are just that :-) policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy # At this point in time, you must list all acceptable 'object' # types. [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert # Passwords for private keys if not present they will be prompted for # input_password = secret # output_password = secret # This sets a mask for permitted string types. There are several options. # default: PrintableString, T61String, BMPString. # pkix : PrintableString, BMPString. # utf8only: only UTF8Strings. # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). # MASK:XXXX a literal mask value. # WARNING: current versions of Netscape crash on BMPStrings or UTF8Strings # so use this option with caution! string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name # we can do this but it is not needed normally :-) #1.organizationName = Second Organization Name (eg, company) #1.organizationName_default = World Wide Web Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) #organizationalUnitName_default = commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 # SET-ex3 = SET extension number 3 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName [ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] # Extensions for a typical CA # PKIX recommendation. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always # This is what PKIX recommends but some broken software chokes on critical # extensions. #basicConstraints = critical,CA:true # So we do this instead. basicConstraints = CA:true # Key usage: this is typical for a CA certificate. However since it will # prevent it being used as an test self-signed certificate it is best # left out by default. # keyUsage = cRLSign, keyCertSign # Some might want this also # nsCertType = sslCA, emailCA # Include email address in subject alt name: another PKIX recommendation # subjectAltName=email:copy # Copy issuer details # issuerAltName=issuer:copy # DER hex encoding of an extension: beware experts only! # obj=DER:02:03 # Where 'obj' is a standard or added object # You can even override a supported extension: # basicConstraints= critical, DER:30:03:01:01:FF [ crl_ext ] # CRL extensions. # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. # issuerAltName=issuer:copy authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] # These extensions should be added when creating a proxy certificate # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName # This really needs to be in place for it to be a proxy certificate. proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] 63765f632f73b0d57424afbf2fb4cb462a2f6f31 2664 2663 2014-04-11T15:26:15Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <code>/etc/ssl/domain</code>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create a list of key revocations root@server:~# echo '01' > domain/crlnumber * Create the <code>/etc/ssl/openssl.cnf</code> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. ==Revoking a certificate== == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME = . RANDFILE = $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids # To use this configuration file with the "-extfile" option of the # "openssl x509" utility, name here the section containing the # X.509v3 extensions to use: # extensions = # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca' and 'req'. # Add a simple OID like this: # testoid1=1.2.3.4 # Or use config file substitution like this: # testoid2=${testoid1}.5.6 #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. #unique_subject = no # Set to 'no' to allow creation of # several ctificates with same subject. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/crl.pem # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert # Comment out the following two lines for the "traditional" # (and highly broken) format. name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options # Extension copying option: use with caution. # copy_extensions = copy # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs # so this is commented out by default to leave a V1 CRL. # crlnumber must also be commented out to leave a V1 CRL. # crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering # A few difference way of specifying how similar the request should look # For type CA, the listed attributes must be the same, and the optional # and supplied fields are just that :-) policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy # At this point in time, you must list all acceptable 'object' # types. [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert # Passwords for private keys if not present they will be prompted for # input_password = secret # output_password = secret # This sets a mask for permitted string types. There are several options. # default: PrintableString, T61String, BMPString. # pkix : PrintableString, BMPString. # utf8only: only UTF8Strings. # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). # MASK:XXXX a literal mask value. # WARNING: current versions of Netscape crash on BMPStrings or UTF8Strings # so use this option with caution! string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name # we can do this but it is not needed normally :-) #1.organizationName = Second Organization Name (eg, company) #1.organizationName_default = World Wide Web Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) #organizationalUnitName_default = commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 # SET-ex3 = SET extension number 3 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName [ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] # Extensions for a typical CA # PKIX recommendation. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always # This is what PKIX recommends but some broken software chokes on critical # extensions. #basicConstraints = critical,CA:true # So we do this instead. basicConstraints = CA:true # Key usage: this is typical for a CA certificate. However since it will # prevent it being used as an test self-signed certificate it is best # left out by default. # keyUsage = cRLSign, keyCertSign # Some might want this also # nsCertType = sslCA, emailCA # Include email address in subject alt name: another PKIX recommendation # subjectAltName=email:copy # Copy issuer details # issuerAltName=issuer:copy # DER hex encoding of an extension: beware experts only! # obj=DER:02:03 # Where 'obj' is a standard or added object # You can even override a supported extension: # basicConstraints= critical, DER:30:03:01:01:FF [ crl_ext ] # CRL extensions. # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. # issuerAltName=issuer:copy authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] # These extensions should be added when creating a proxy certificate # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName # This really needs to be in place for it to be a proxy certificate. proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] af62192ffa08cdb0d53ce3f6c95d59ec51918c40 2665 2664 2014-04-11T15:55:44Z Neil 2 /* Revoking a certificate */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <code>/etc/ssl/domain</code>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create a list of key revocations root@server:~# echo '01' > domain/crlnumber * Create the <code>/etc/ssl/openssl.cnf</code> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. ==Revoking a certificate== When certificates expire or become compromised, they can be revoked. * Revoke a certificate with the command: root@server:/etc/ssl# openssl ca -revoke domain/certs/squirrelmail-cert.pem * Once you've done all the revocations you need to, generate the certificate revocation list (CRL): root@server:/etc/ssl# openssl ca -gencrl -out domain/root.crl * You can check the contents of the revocation list with: root@server:/etc/ssl# openssl ca -gencrl -out domain/root.crl * Copy the new CRL to the correct directory on the wweb server: root@server:/etc/ssl# cp domain/root.crl /var/www/domain.tld/root.crl You can then generate new certificates to replace the revoked ones. == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME = . RANDFILE = $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids # To use this configuration file with the "-extfile" option of the # "openssl x509" utility, name here the section containing the # X.509v3 extensions to use: # extensions = # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca' and 'req'. # Add a simple OID like this: # testoid1=1.2.3.4 # Or use config file substitution like this: # testoid2=${testoid1}.5.6 #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. #unique_subject = no # Set to 'no' to allow creation of # several ctificates with same subject. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/crl.pem # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert # Comment out the following two lines for the "traditional" # (and highly broken) format. name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options # Extension copying option: use with caution. # copy_extensions = copy # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs # so this is commented out by default to leave a V1 CRL. # crlnumber must also be commented out to leave a V1 CRL. # crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering # A few difference way of specifying how similar the request should look # For type CA, the listed attributes must be the same, and the optional # and supplied fields are just that :-) policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy # At this point in time, you must list all acceptable 'object' # types. [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert # Passwords for private keys if not present they will be prompted for # input_password = secret # output_password = secret # This sets a mask for permitted string types. There are several options. # default: PrintableString, T61String, BMPString. # pkix : PrintableString, BMPString. # utf8only: only UTF8Strings. # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). # MASK:XXXX a literal mask value. # WARNING: current versions of Netscape crash on BMPStrings or UTF8Strings # so use this option with caution! string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name # we can do this but it is not needed normally :-) #1.organizationName = Second Organization Name (eg, company) #1.organizationName_default = World Wide Web Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) #organizationalUnitName_default = commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 # SET-ex3 = SET extension number 3 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName [ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] # Extensions for a typical CA # PKIX recommendation. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always # This is what PKIX recommends but some broken software chokes on critical # extensions. #basicConstraints = critical,CA:true # So we do this instead. basicConstraints = CA:true # Key usage: this is typical for a CA certificate. However since it will # prevent it being used as an test self-signed certificate it is best # left out by default. # keyUsage = cRLSign, keyCertSign # Some might want this also # nsCertType = sslCA, emailCA # Include email address in subject alt name: another PKIX recommendation # subjectAltName=email:copy # Copy issuer details # issuerAltName=issuer:copy # DER hex encoding of an extension: beware experts only! # obj=DER:02:03 # Where 'obj' is a standard or added object # You can even override a supported extension: # basicConstraints= critical, DER:30:03:01:01:FF [ crl_ext ] # CRL extensions. # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. # issuerAltName=issuer:copy authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] # These extensions should be added when creating a proxy certificate # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape's comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName # This really needs to be in place for it to be a proxy certificate. proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] d4815de8710926630c81dfb54738044eb81c50b3 2666 2665 2014-04-11T16:05:30Z Neil 2 /* Contents of /etc/ssl/openssl.cnf */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <code>/etc/ssl/domain</code>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create a list of key revocations root@server:~# echo '01' > domain/crlnumber * Create the <code>/etc/ssl/openssl.cnf</code> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. ==Revoking a certificate== When certificates expire or become compromised, they can be revoked. * Revoke a certificate with the command: root@server:/etc/ssl# openssl ca -revoke domain/certs/squirrelmail-cert.pem * Once you've done all the revocations you need to, generate the certificate revocation list (CRL): root@server:/etc/ssl# openssl ca -gencrl -out domain/root.crl * You can check the contents of the revocation list with: root@server:/etc/ssl# openssl ca -gencrl -out domain/root.crl * Copy the new CRL to the correct directory on the wweb server: root@server:/etc/ssl# cp domain/root.crl /var/www/domain.tld/root.crl You can then generate new certificates to replace the revoked ones. == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # HOME = . RANDFILE = $ENV::HOME/.rnd oid_section = new_oids #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/root.crl # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name organizationalUnitName = Organizational Unit Name (eg, section) commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. basicConstraints=CA:FALSE nsComment = "OpenSSL Generated Certificate" subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer keyUsage = digitalSignature,keyEncipherment extendedKeyUsage = serverAuth crlDistributionPoints = URI:http://domain.tld/root.crl subjectAltName = @alt_names [alt_names] DNS.1 = domain.tld DNS.2 = *.domain.tld [ v3_req ] basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always basicConstraints = CA:true [ crl_ext ] authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] basicConstraints=CA:FALSE nsComment = "OpenSSL Generated Certificate" subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] 21ee814855e1ac82144b6571d3418d509a77c4ab 2667 2666 2014-04-11T20:12:07Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <code>/etc/ssl/domain</code>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create a list of key revocations root@server:~# echo '01' > domain/crlnumber * Create the <code>/etc/ssl/openssl.cnf</code> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. ==Revoking a certificate== When certificates expire or become compromised, they can be revoked. * Revoke a certificate with the command: root@server:/etc/ssl# openssl ca -revoke domain/certs/squirrelmail-cert.pem * Once you've done all the revocations you need to, generate the certificate revocation list (CRL): root@server:/etc/ssl# openssl ca -gencrl -out domain/root.crl * You can check the contents of the revocation list with: root@server:/etc/ssl# openssl ca -gencrl -out domain/root.crl * Copy the new CRL to the correct directory on the wweb server: root@server:/etc/ssl# cp domain/root.crl /var/www/domain.tld/root.crl You can then generate new certificates to replace the revoked ones. ==Viewing the contents of a certificate== root@server:/etc/ssl# openssl x509 -in domain/certs/squirrelmail-cert.pem -text == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # HOME = . RANDFILE = $ENV::HOME/.rnd oid_section = new_oids #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/root.crl # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name organizationalUnitName = Organizational Unit Name (eg, section) commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. basicConstraints=CA:FALSE nsComment = "OpenSSL Generated Certificate" subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer keyUsage = digitalSignature,keyEncipherment extendedKeyUsage = serverAuth crlDistributionPoints = URI:http://domain.tld/root.crl subjectAltName = @alt_names [alt_names] DNS.1 = domain.tld DNS.2 = *.domain.tld [ v3_req ] basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always basicConstraints = CA:true [ crl_ext ] authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] basicConstraints=CA:FALSE nsComment = "OpenSSL Generated Certificate" subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] a191db3db5734baab2de5103fa179218d047cf09 0 1526 2668 2621 2014-06-14T17:10:09Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install ubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install Scikit-learn user@desktop:~$ sudo pip install scikit-learn user@desktop:~$ sudo pip3 install scikit-learn * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] f9b1291fc4e1258766f1ce89f8597803fb7870df 2670 2668 2014-06-22T11:06:18Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file: if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install Scikit-learn user@desktop:~$ sudo pip install scikit-learn user@desktop:~$ sudo pip3 install scikit-learn * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 4cc9273a9fc4b3a9646227132d1e1ae524da5e17 2674 2670 2014-06-23T08:38:57Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: user@desktop:~$ sudo rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install Scikit-learn user@desktop:~$ sudo pip install scikit-learn user@desktop:~$ sudo pip3 install scikit-learn * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 5b7609cb8e9f3074b7c7917559999c8b82d7ba07 2675 2674 2014-06-23T08:56:10Z Neil 2 /* Install Ruby */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas * Install Scikit-learn user@desktop:~$ sudo pip install scikit-learn user@desktop:~$ sudo pip3 install scikit-learn * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] bf99ad0836f6b728f5a0febf94d3eaa86353c576 2676 2675 2014-06-23T14:54:58Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] ea6628626745842fa4ab86746b2488c231c6c457 2677 2676 2014-06-24T13:22:09Z Neil 2 /* Install Sublime Text */ Added Ibus section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) ==Change IBus default shortcut== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. [[Category:Server setup]] 9f28871327bd2fb11b6e97e7fb87e745fc671912 2678 2677 2014-07-11T11:25:55Z Neil 2 Moved IBus section, added section on disabling it. wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. (Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) [[Category:Server setup]] 99590d93d0b19a601903860573eba3788fe3e078 2679 2678 2014-07-11T16:05:38Z Neil 2 /* Install Sublime Text */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] (and linter packages for [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby] and [https://sublime.wbond.net/packages/SublimeLinter-pylint Python]), [https://sublime.wbond.net/packages/SublimeCodeIntel Code Intel], and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements]. :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Make Sublime use the Source Code Pro lines by adding these lines in Preference | Settings - User "font_face": "Source Code Pro", "font_size": 10 [[Category:Server setup]] 7d0e4d517b462347d80dfa9ab19100adc5174b70 2680 2679 2014-07-11T17:36:42Z Neil 2 /* Install Sublime Text */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] (and the colour scheme packages [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] and [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow]. :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select 'Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } [[Category:Server setup]] e50ce9e626319ea9d507d18d76d632dac77f4e65 2681 2680 2014-07-11T17:38:09Z Neil 2 /* Install Sublime Text */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] (and the colour scheme packages [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] and [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow]. :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } [[Category:Server setup]] 84ecbbf62efe48876c298a1c19cd44ba89787c26 2682 2681 2014-07-13T13:02:40Z Neil 2 /* Install Sublime Text */ Added linters wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] (and the colour scheme packages [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] and [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow]. :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } [[Category:Server setup]] d58a5f5f2b581c166a1bd4b0ab0f04ab6c7cc4a6 2683 2682 2014-07-13T13:23:24Z Neil 2 /* Install Sublime Text */ Added pylinter settings wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] (and the colour scheme packages [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] and [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow]. :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } [[Category:Server setup]] c99930204b4807df422e7509fa3252d5db450919 2684 2683 2014-07-13T22:43:19Z Neil 2 /* Install Sublime Text */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete], [https://sublime.wbond.net/packages/GitGutter Git Gutter], [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] and [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] (and the colour scheme packages [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] and [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow]. :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } [[Category:Server setup]] 14564ad3549753df906ec2a0f5f8b23cd6c912f2 2685 2684 2014-07-16T21:24:37Z Neil 2 /* Install and configure Sublime Text */ Added git ST package, some formatting tweaks wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } [[Category:Server setup]] 18e92a2098c131fee5e7fbbd0fe76085ab386fcb 2686 2685 2014-07-20T15:03:49Z Neil 2 Added ksysguardd on server wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -e [0-9] | xargs sudo apt-get remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. [[Category:Server setup]] 8aea61ed61b104b030f7674fe4232a05749bae56 2695 2686 2014-09-07T13:40:27Z Neil 2 /* Remove old kernels */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. [[Category:Server setup]] 8d2080ffee913d4e87d393b3067a2a05052fcbec 0 1543 2669 2564 2014-06-20T14:20:13Z Neil 2 Added ufw section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. ==Server== === Remove ufw === The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. === Add a bespoke iptables firewall === * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> ==Desktop== The routing tables on the router mean that people can't get direct access to the desktop machine. In this case, <code>ufw</code> is sufficient. ===Set up ufw=== Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 739609d8e7329e3cd9048309d06fb869a1d9b532 0 1630 2671 2543 2014-06-22T15:37:27Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity biblatex blueman build-essential calibre cheese cmake couchdb-bin create-resources cython dos2unix dpkg-dev easytag eog fbreader ffmpeg fgetty finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing fop gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gmusicbrowser gnome-games gnubg gnuplot-qt gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve gucharmap guvcview hal-info haskell-mode haskell-platform haskell-platform-prof llvm hplip-dbg hplip-gui html2text ibus idle idle3 inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs normalize-audio octave ogmtools openshot p7zip pandoc paprefs pastebinit pavucontrol pavumeter pdftk perlmagick phonon phonon-backend-gstreamer pitivi postgresql postgresql-client pychess python-all python3-all r-base r-recommended samba sbcl scratch screen scribus scribus-doc scribus-template seahorse shotwell slime sni-qt software-center software-properties-gtk sound-juicer sqlite3 stellarium swi-prolog tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science thunderbird tk-dev tk tofrodos traceroute tuxpaint tuxpaint-config units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xsane xtightvncviewer zenity 7d24a262963e870185bc2a882486004a27ab3144 2672 2671 2014-06-22T16:46:46Z Neil 2 Trimmed the package list wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity build-essential calibre couchdb-bin create-resources dos2unix fbreader finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gnome-games gnubg gnuplot-qt # Possible conflict with gnuplot-nox gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve hal-info haskell-mode haskell-platform haskell-platform-prof llvm ibus inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs npm normalize-audio octave ogmtools openshot p7zip pandoc pavucontrol pavumeter pdftk pitivi postgresql postgresql-client r-base sbcl scratch screen scribus scribus-template seahorse shotwell slime sqlite3 stellarium swi-prolog tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science tk-dev tk tofrodos traceroute units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xtightvncviewer zenity Packages which were either already installed by default, or no longer necessary. # biblatex # blueman # cheese # cmake # cython # dpkg-dev # easytag ? # eog # ffmpeg # fgetty # fop # gmusicbrowser # gucharmap # guvcview # html2text # idle # idle3 # paprefs # pastebinit # perlmagick # python-all # python3-all # r-recommended # samba # sni-qt # software-center # software-properties-gtk # sound-juicer # thunderbird # tuxpaint # tuxpaint-config # xsane de92da4b4308f369777043824acb9bc0af7aa7d0 2673 2672 2014-06-23T08:26:55Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity build-essential calibre couchdb-bin create-resources curl dos2unix fbreader finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gnome-games gnubg gnuplot-qt # Possible conflict with gnuplot-nox gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve hal-info haskell-mode haskell-platform haskell-platform-prof llvm ibus inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs npm normalize-audio octave ogmtools openshot p7zip pandoc pavucontrol pavumeter pdftk pitivi postgresql postgresql-client r-base sbcl scratch screen scribus scribus-template seahorse shotwell slime sqlite3 stellarium swi-prolog tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science tk-dev tk tofrodos traceroute units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xtightvncviewer zenity Packages which were either already installed by default, or no longer necessary. # biblatex # blueman # cheese # cmake # cython # dpkg-dev # easytag ? # eog # ffmpeg # fgetty # fop # gmusicbrowser # gucharmap # guvcview # html2text # idle # idle3 # paprefs # pastebinit # perlmagick # python-all # python3-all # r-recommended # samba # sni-qt # software-center # software-properties-gtk # sound-juicer # thunderbird # tuxpaint # tuxpaint-config # xsane d9b8eee8bdee36b9a1eb05537c875201789b7263 2687 2673 2014-07-24T15:52:22Z Neil 2 Added exfat-utils wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity build-essential calibre couchdb-bin create-resources curl dos2unix exfat-utils fbreader finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gnome-games gnubg gnuplot-qt # Possible conflict with gnuplot-nox gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve hal-info haskell-mode haskell-platform haskell-platform-prof llvm ibus inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs npm normalize-audio octave ogmtools openshot p7zip pandoc pavucontrol pavumeter pdftk pitivi postgresql postgresql-client r-base sbcl scratch screen scribus scribus-template seahorse shotwell slime sqlite3 stellarium swi-prolog tesseract-ocr texlive-bibtex-extra texlive-pictures texlive-science tk-dev tk tofrodos traceroute units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xtightvncviewer zenity Packages which were either already installed by default, or no longer necessary. # biblatex # blueman # cheese # cmake # cython # dpkg-dev # easytag ? # eog # ffmpeg # fgetty # fop # gmusicbrowser # gucharmap # guvcview # html2text # idle # idle3 # paprefs # pastebinit # perlmagick # python-all # python3-all # r-recommended # samba # sni-qt # software-center # software-properties-gtk # sound-juicer # thunderbird # tuxpaint # tuxpaint-config # xsane 58b5d8b2bc40b87b20ea888dda593e555c58fa97 0 1 2689 2550 2014-08-08T09:47:44Z Neil 2 /* Neil */ Added GPG public key wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == {| align="right" {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] |} Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. fc461fbe5decbc1e27c61a4bc1279d38174fbcce 0 1587 2690 2491 2014-08-08T09:49:05Z Neil 2 /* Import a secret/public key pair from another source */ Changed default public key server wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[VNC setup|VNC]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. == Import a secret/public key pair from another source == === Add another identity to a key === === Publish a public key === I genererally use the [https://keyserver.pgp.com/ PGP Public Directory] for keys. === Sign someone else's public key === === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == There is a plugin for this, but it hasn't been updated in a long time and no longer works. == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] eedaa55a748aae8bb835754182a9d03af0cfcd93 2693 2690 2014-08-08T17:26:39Z Neil 2 /* Import a secret/public key pair from another source */ Added Android section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[VNC setup|VNC]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. === Publish a public key === I genererally use the [https://keyserver.pgp.com/ PGP Public Directory] for keys. === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. ===Integration in Android=== * Install [https://play.google.com/store/apps/details?id=org.thialfihar.android.apg AGP]. * Copy the <code>pubring.gpg</code> and <code>secring.gpg</code> onto the device. * Import the secret and public keys. Note that you should only import a few at a time: the app seems to baulk at importing more than about eight keys at a time. * [https://play.google.com/store/apps/details?id=com.fsck.k9 K9 Mail] should now provide "Sign" and "Encrypt" checkboxes when composing new messages, and will verify signed messages you receive. (It doesn't seem to decrypt encrypted messages.) == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == There is a plugin for this, but it hasn't been updated in a long time and no longer works. == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] 411bd12267545bc8f9e37c82b5c6d0ec9d86aa10 2694 2693 2014-08-08T18:50:08Z Neil 2 /* GnuPG for Email */ Added Thunderbird section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[VNC setup|VNC]] |align="center" width="50%"| None |} Cryptography is important to protect email and other files. Here I'll talk about using cryptography for signing and encrypting email. * Check GnuPG is installed (it should be anyway): root@desktop:~# apt-get install gnupg gnupg-agent gpgsm :and install a certificate revokation checker and a graphical front end: root@desktop:~# apt-get install dirmngr kgpg == GnuPG for Email == (Note: these instructions may be a bit inaccurate as I wrote this up some time after doing it. Some details may be wrong.) When KGpg starts, it appears as a small padlock icon in the system tray. Don't expect a full window and get disappointed when KGpg apparently doesn't work! === Create a key === KGpg is also not reliable when it comes to creating keys. Do key creation from the command line. Creating a key is simple: user@desktop:~$ gpg --gen-key Accept the defaults and use a strong passphrase. === Publish a public key === I genererally use the [https://keyserver.pgp.com/ PGP Public Directory] for keys. === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the key. ===Integration into Thunderbird=== Use the Enigmail plugin. root@desktop:~# aptitude install enigmail Follow the instructions when you restart Thunderbird. ===Integration in Android=== * Install [https://play.google.com/store/apps/details?id=org.thialfihar.android.apg AGP]. * Copy the <code>pubring.gpg</code> and <code>secring.gpg</code> onto the device. * Import the secret and public keys. Note that you should only import a few at a time: the app seems to baulk at importing more than about eight keys at a time. * In [https://play.google.com/store/apps/details?id=com.fsck.k9 K9 Mail], save any account settings you may already have, remove it, and reinstall. K9 Mail should now provide "Sign" and "Encrypt" checkboxes when composing new messages, and will verify signed messages you receive. (It doesn't seem to decrypt encrypted messages.) == x509 certificates for Email == The [http://www.thawte.com/secure-email/web-of-trust-wot/ Thawte Web of Trust] (WoT) is a free service for asserting your identity. First, you register for a free email signing certificate from Thawte. You then have your identity verified by various Thawte WoT notaries (all volunteers). That allows you to include your name in your certificate. One that is done, you can sign and ecrypt emails with that certificate. === Get a certificate === * Register with Thawte to receive your email signing certificate. Don't create one yet. * Find WoT notaries in your area and have them corroborate your identity. Each corroboration will give you some trust points. * Once you have 50 trust points, you can create a certificate that includes your name. Do so now and import it into Firefox. === Import the certificate into GnuPG SM === * In Firefox, export the certificate to somewhere on your local disk (Edit | Preferences | Advanced | Your Certificates | Backup). Let's call the saved file <code>thawte-wot-certificate.p12</code>. You will need to provide a password for access to this saved certificate file. * Convert the p12 file to a PEM file: Now you have to get the certificate's keys into GpgSM's keyrings. user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out thawte-wot-certificate.pem -nodes :(give the password you specified when you exported the certificate) * Extract your private key: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.pem -export -out my-key.p12 -nocerts -nodes :(give the password you specified when you exported the certificate and specify a password to protect your private key file) * Import your private key into your private keyring: user@desktop:crypto$ gpg-agent --daemon gpgsm --call-protect-tool --p12-import --store my-key.p12 :(give the password you specified when you exported the key and specify a password to protect your private key in GpgSM's keyring) * Extract your public key from the certificate file: user@desktop:crypto$ openssl pkcs12 -in thawte-wot-certificate.p12 -out certs.pem -nokeys :(give the password you specified when you exported the certificate) * Import the certificates into your public keyring. user@desktop:crypto$ gpgsm --import certs.pem * Tell GnuPG that you trust Thawte. Get the list of keys and note which have Subjects that relate to Thawte: user@desktop:crypto$ gpgsm -kv /home/user/.gnupg/pubring.kbx ----------------------------- ID: 0x1839990F S/N: 0D Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail Issuing CA/O=Thawte Consulting (Pty) Ltd./C=ZA validity: 2003-07-17 00:00:00 through 2013-07-16 23:59:59 key type: 1024 bit RSA key usage: certSign crlSign chain length: 0 fingerprint: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F ID: 0xA4EB0085 S/N: 00 Issuer: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com Subject: /CN=Thawte Personal Freemail CA/OU=Certification Services Division/O=Thawte Consulting/L=Cape Town/ST=Western Cape/C=ZA/EMail=personal-freemail@thawte.com validity: 1996-01-01 00:00:00 through 2020-12-31 23:59:59 key type: 1024 bit RSA chain length: unlimited fingerprint: 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 * Copy the fingerprints into <code>~/.gnupg/trustlist.txt, with <code> S relax</code> appended to each of them: BC:F0:3A:B1:BD:9A:08:9B:EB:46:8D:AF:99:47:5E:83:18:39:99:0F S relax 20:99:00:B6:3D:95:57:28:14:0C:D1:36:22:D8:C6:87:A4:EB:00:85 S relax * There is a problem with Thawte certificates and their certificate revokations, so you need to prevent gpgsm from looking for key revocation certificates. Create <code>~/.gnupg/gpgsm.conf</code>, containing the line disable-crl-checks === Testing === * Create a test file, then sign it: user@desktop:~$ gpgsm --detach-sign test-file > test-file.sig * Check the signature: user@desktop:~$ gpgsm --verify test-file.sig test-file === Integration into KMail === * Open the KMail settings diaglog: KMail | Settings | Configure KMail * Select the 'Identities' tab and select the one you want. * Press the 'Modify...' button and select the 'Cryptography' tab * Add the key for both Signing and Encryption You should now be able to sign and encrypt messages with the WoT certificate. == Allowing GnuPG signing in Squirrelmail == There is a plugin for this, but it hasn't been updated in a long time and no longer works. == Getting trusted keys == ''Insert descriptions and links to CAcert and GSWoT'' == See also == * [http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto.html GnuPG Howto] * Some [http://www.mew.org/feature/smime.html.en notes on installing S/MIME certificates with GpgSM] [[Category:Server setup]] e1b83fed472d3045fa7430bca7ef1e34f8aa0b42 0 1642 2692 2014-08-08T11:08:42Z Neil 2 Created page with "<pre> -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1 mQINBFPjtPsBEACxicibg9FGzUsWDvPxHlFfCN7dtxl+JM2tYm+kmO7kGT1v00/W xk94CGot4ZBm+d0hdKDo2V1CzP0lTfXCBZg8wHxJYt8KK+jsZa6…" wikitext text/x-wiki <pre> -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1 mQINBFPjtPsBEACxicibg9FGzUsWDvPxHlFfCN7dtxl+JM2tYm+kmO7kGT1v00/W xk94CGot4ZBm+d0hdKDo2V1CzP0lTfXCBZg8wHxJYt8KK+jsZa64VopIzL3ZL2H7 NhAt8Zs7uLwVC/9IGmCtCrZ2n4h7JD1we+ZI645ULacw96DSewP25dxIk/SbdI2d lTq27tBXeS6i/IF6gNph4RTNPqVeZnTsi8A7lxKaUZnB7tA7TbjfyCgV/QFR5cTO gWH897LC2/24IQZZRzoL6tIk4XYy/WOvWi/Usd94slQjcUoq2Wx3t96XCqHJT2sE FJMTfGGmtVIKnAtvIZw//84BpaUfKk37DD1e75eyTgUNtqo2QG8luU8m5CRtC1w3 zh7ZH/JBvdbxiRR38dl5CLeYvBWl500sKFtn3fsik9T/1oyDQAPKi5LKZwT2Pf9L roY7SLZ4pGrrP9mYe+Q740mc/3ZHvl4NTyibh90/WyFAujlS2JXevoP5TTbA4t/T +6kg65IcE58Gn898WFiwB+R+jTaoyLaf0XOqgsApLdrKPcrzBxfRCN10a18K+Y34 4UO4h1AIpcivQlYaHDY2MBcncSYv37WPJ9iRKlWafgaVK3NGEmxud0fL74PLXltQ 6CMUBMlXa1O23eXDGJVaCNPEznsI20A11pSdQEBhrFauyVzvMqi2e9PxVwARAQAB tDROZWlsIFNtaXRoIChodHRwOi8vd3d3Lm5qYWUubWUudWspIDxuZWlsQG5qYWUu bWUudWs+iQJBBBMBAgArAhsDBQkFo5qABgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIX gAUCU+PqLQIZAQAKCRCTwdntOz4G6mz7EACBkta5mggeiPboEQ2CUCNMJFJAunvL XIixp3OhAShhsGOAwJ/uypHg5K1jlYbJj6RXv5+Yb1KnDePe9nJMdf8h/kgpxiAC f5COOj2gkW2ROXZYhU8x6DuDsIMiE//XZDdvEKTGmvEf4HSDhRw/TJ/T0OrSId+9 4nIEnKMxV08fXcMI+v+CMCnP2Se52yiQZPPQA+wDF3KpKng6PyRM9WiJKF9ZFAQE mm39PxLqPXYx0yhyQYLQs0Z8mAkm/7rqoP8mG7L4A27cYrXn6W53H/hLjPJOGr3+ NOD1Ese1IRdM5aE2yQKl3+1wGGADN3ATKFb0eZwmMl5DNqUg2SRxZ2WmS8K+3Fn1 xlZOGQovBsMiSjqRGjkdEvlFh+d6efUeeFNhPZGByt22fQLd7sUvK3Qfh44hvHJc 3GqdA/I0DGDFN1w0VJp87+wIcAbVjwrxrQ8U5G+MsI3gbWkFYUw1p3u9ustPp9CL cw8OUQUPG7bE0kNKaEXnatiaINl21Lpjc6I1a+CpVq+7bEHHfa8vvmKQAaj0dsSg SAdUAemzvY3wgaRpZfN5Ljq5r/H1bxi/mgTHaUdrAItko7tDjy+oMXxi04MaM7u/ I+2IPRlWKAIfYY1naWpi4CUJebGAPaCFSj+zg17BvaUVFFX4Ru10Px12vdz0+T7C ytugiNVCBkPpL7Q3TmVpbCBTbWl0aCAoaHR0cDovL3d3dy5uamFlLm1lLnVrKSA8 bmVpbC5kMUBuamFlLm1lLnVrPokCPgQTAQIAKAUCU+Pp9wIbAwUJBaOagAYLCQgH AwIGFQgCCQoLBBYCAwECHgECF4AACgkQk8HZ7Ts+Bup2gw//WJAaSi65wkKykK4g LMsII+U9dUH4/qBdHGi/mqDiHaJ4E87n0OXx3y0eFoD7it9H8VrD0dnkRa1tWGZH O4KqeUuLAT6x4maWcfJ3Q2lpNtzXryfVgx6qQv99UoSwtP76v8H9KZEgGW8vUU32 ZEofnLXO95ei1fYHIY1HKfiMRr7WEc2cteHjNPK+juUAPNtbSVmMJ51sTQ3aqbNE us2yYTKN8aeR8uWk+PK0zWv4H/sGtjxumJQ5WvNni9SbKPyoxWs5sYhfvaoWWTia cAku8O5nHlcVTA6s9CkxAqZzQaMwTEYwg8EjbAzBoRR+GGa/iRN7PNPqTahFJxcE AIPQ92vMKGCYVnmAYoozCEcUVKz6J93ZDUZVTMpssirsjsS29siCScVHJIJd9Bwm Wek2MppF3BkwCa4nxSEbYIT7XSC35EbwLfrS1EiVSRvfozNi6q6VftIQmQR6ODbq vaD8JoUxVGRGGv+wpcKHAFzbxRnHxYIr7rpyO4BTmZJoJ8hiP8bp9U7G4ckgCFSy jt+KhbEQKBIPxNu8jnPEreqIo3X4QPqeF2qrNppn4p+JwfYl4NBBNOF/gB/bkiHU Xqz//bsmmyIjRaUnbTWOZEr5Munu9gfeVnPD1ejgmyoYIL8R9eSJCAAeHnfghVrl VVWcTaOSiDvcNSwg0FUe7+90gMO0Ok5laWwgU21pdGggKGh0dHA6Ly93d3cubmph ZS5tZS51aykgPG5laWwueWFob29AbmphZS5tZS51az6JAj4EEwECACgFAlPj6gwC GwMFCQWjmoAGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJPB2e07PgbqAeIP /R/wUZc0MwTkPuEam9C+8D1hNBjMHjkVw2grfqqdRFGx15Y+j7CtYXbloJ4ofFYD M+GiKCjeHH53pM/071NJoKZWvxC0pWeTEx+F/6Kg8LMneRnndx/O5P6dHigoxG5m GXh2TTEmIiTdvSKh5+LYkobOvL9iPymqD988gpm/HJIK0AJj97YUKoxPjVTOhuP/ 0ZDRsc8TsKwK5+QjhR2rY3yCAeLXvaOUfqoyWst0BuS/klFpHCx6dxPIij8CZMIe DPBn/mRFqJxUJRYuzF2H151DiYGY2zMNmts51c/FpZHIOSIyv7MofinYLkLRFFD0 bNf1e4iwIvq9GGoQmOM8qFAFpT7gGI8N3IAotDuJpyHixCoQsMGMKm9p9zz6yYga eiDahomtscGh4dcDgneUa1X1bMQ84vyQR3XbjSzmLs75uM/B7VBUII3OPNALSQkY SHyxGH7t4ftMdr1eQYX7bsJbe0/+oyWL12/TxtznJW8vP+VjbZV7zotoXB9IxjaW r2IG2vZlJ3y0acPKRGWia92DEfb8JMAJwvHvC/G13bWI6tdugYqUSOuWgNU+nRO4 mbdrp9WnshD22FrAXa+0XwSxcN0O9nKkTlBTiK74LsBlvoqvo/NRELvhbc6yXTzz Ge10ty4I3Ba/Bo0A5HKjIEy9SZm9fAXKIGzVdrIro7k5tD1OZWlsIFNtaXRoICho dHRwOi8vd3d3Lm5qYWUubWUudWspIDxuZWlsLmNvZGVjbHViQG5qYWUubWUudWs+ iQI+BBMBAgAoBQJT4+sEAhsDBQkFo5qABgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIX gAAKCRCTwdntOz4G6rKhD/9FR5+mqeskp+zaVbNAilph5YQ25wPi2coHfxguyUmq sgsOaurR6mKDtY5nQG+n7F/5eAwduhXBOWMSZmtYTsx/ng2mRm3/Q9vJ8r2WCAEX 0iuVGowI34jsjMVKJYCgGQKCapSFiDkLv04bE89jAjqupobYL/LB3mEvHQb5WRIj 3ovAZ1yo0pQVkNe6zLSG+LGXqDByjIlUEN9Zi9c62Gd5hUBdFJXjnL5D+Qt1xqlw tm0ofB9ZSRBXDkK4+TvrTxCbr9fxYJ1l0jd2V59TT1dBPXzDImnWR+Pk0BL+djOW neUr8L5VOFb7Dozp9n20c0EGXmWqC2F4A6I3pSdG0Fj9j8gFxu4VvYw8ab47fUqS Kb2FbAi6irXQDuLCu6lDMnWd/eCKZ6y+SgPYgnhbPU0HWUgH+5rf/OIFjyPbRnVA XZgmQTiDJrDBoVcDS+UwHy8hyiKOSEp3PaNGuGTpVF6U8voDEja1f7zxv5bJ3bQH n2zChAmaiK1e1vQVP4UazEKWoH3O07xoQBm8UXof6gclKMyygCuGGe+qkS3x9mRD emFpFGGhPcmptUEt/Qc+5lp4ncJPcGQL8eO3GbRvlJJCyQCrJJ+BhkBGkxTDkZIS p9Z5EdhIV3C7itXbEIm4VdnGp3F1npEFzfcHdVo6sJOEc1xdt2XVEbQ+oMeC4LGZ uLQ6TmVpbCBTbWl0aCAoaHR0cDovL3d3dy5uamFlLm1lLnVrKSA8bmVpbC5pZG5l dEBuamFlLm1lLnVrPokCPgQTAQIAKAUCU+PrHAIbAwUJBaOagAYLCQgHAwIGFQgC CQoLBBYCAwECHgECF4AACgkQk8HZ7Ts+BupHiBAAnSEm9dDeGCLN/ZosIo0Ovl6W P/w2xXX98kJttTzDmkcVkoitS2/twsAZaWifeqIueyYoGlq+bzF8hs5Pw8gj8aJe egbcryBkOf0pXFgwJHk3NfmHaHLabamXe6us3r+nSgg1UCHcih5RLKecT+gVwCdm pRVW7fRJKWhmyN4rvAvXFkbbN+5u9LsRL4KIHWycMxMsIFSGJKaRbq0uAsW0vM5z y7/3C2aV/ORFU3Tyz8jwo45OXeP/owA/fQRDUYIAYcLikcOaxo2bhW94kJD5RSH8 v4AePOxU5a46GqU+eQhzrphI2Rzy4YgfgJtT0MuvRhcGiLfe8VgBP7bVFzKspe// 352FprtEddu4F4kmfoNsylpD3PTcYSP8AvzSBdJzNhwM1Hw/LnlPEHJ5UMyXykCN VmFP/sYf/IAEAz9PZdYefde/Vg4f8/cZzmVoqgbHW9bf+shXOYl3AQ0XGeX3EMzH vqtC78YOtO069MukB09A76M25Gg+h62mCV6akM1AizrBojdhb6Vg6z32wbPyBbhA 3SZQWri8jw72Diy0ABtnBcWBONeaNdPZ2xPslne43baRK/qkboN7ThPpkuXa4ZuQ ZttnQAyHGbKrk3IT+qiMQQWF78b9jQjkYogF84ZJv06yYjuCKnSEFt/bwdfRfFAL rp/Q4NhyEbWyXgxDiXK0N05laWwgU21pdGggKGh0dHA6Ly93d3cubmphZS5tZS51 aykgPG5laWwubXBAbmphZS5tZS51az6JAj4EEwECACgFAlPj6zUCGwMFCQWjmoAG CwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJPB2e07PgbqhAAP/1H3P+HVRLBE o1hGRpEMKEjH5axO2+pCz8FKSPpNGo54PhOQBh5Gt5Dh9E6xnYgooAOS7+/4m9ds w7NNYfp2lH6c1yVwkY6nH2IBS/pqqWvGyqwGsnvtN0h+JU8tP416+EcoLQSEL1YD tw9/uqyZLPP5CgQFtaOL2AjLlge8P8ls1tQqk/n5Wl7jbfgb0s++kq79JNur9C7S PSfkM2Drw/rVAIUV1HvL1sJxet7pZ6slWl693gO8zDmLwlAO7E9hfO2EsCw9XEKJ CTMxjq4KfRqhaNFMFG9+vLS3fnDH57xq3x1yXpL3WVQXQloqeRDLRULSVe9r/Vn3 NiN3Va6QRfRGyM0lvfSZSEEMv3dsh/zVkdA7LpUxOHWCBnUl3KhMjjKglr8MXsty juDefn3uczsyDDlqykQ7E/t68//ogz7nQILdn39UD0lNxJUuD0AaipfeNj6ip6vz tEEHRk1b2hFLIBIzuBoKnMxWqsmB2epb+3k4jda0ZfzC7lahtw70zHUZ/Cbm6blp 3Kbj6StCf3lUwuHqM0UQAX9DqJ1SM+prDysepOi+6ICZym2Ggiql8SGAxEli4JAM kdujSUQpQG2kjdAai62IrL/g10XoOPQ0MNdYa70eWpGN7DjeoTqKgKAfWqrQZ1RZ wWB74a2qpnTKArWajSOunx8bCg9UYtnYtDhOZWlsIFNtaXRoIChodHRwOi8vd3d3 Lm5qYWUubWUudWspIDxuZWlsLnNiZUBuamFlLm1lLnVrPokCPgQTAQIAKAUCU+Pr jAIbAwUJBaOagAYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQk8HZ7Ts+Buow Qw//QXHoVJVwmezYqeV7PBgyixuNfDfTMUOZyM4CvRErIRXH+ElyjNpeYf1bJvt2 EgfUtNjXa39ISteG7WJ6P5L1PWMuhppeA1ZU6cmlRvUQ7/ylABIhnv7dmusdbDZr Qz6+HqAxanEx/NL0lm2SFMWSw7E3KBatmcmAqLUu5OB2OYeqDwLEAXusuhEM3wyB 4AWkhC/Iy/SRDgcyERPESA2eXRcEHoe1rHhacJUCwww81YcL9PpovCNqU/zL6KaZ j0WwJze8qMzyDHA69UgxoUVVD0fCDtCBX66KpYu2FRKrYDySZ2RYbj5Zy0ZtEor4 gmhOdZTIuRbPzlTdacpLP9SJmtQjtuHWTwBv9eJvrcKlIkmciKKZS82RnQXGb2A0 dmON5ksjiipuXVQrZsfhBNrBSGVrMoj6A69dZyjTKNaAnymBfgFtrixuCfxptrKt RsvnW5tTmmgsL4lELOAw//0gf/WhnAaMVDHcwCOQRrJV/U99obyjKrwcncIayWC1 NPv/GQSiOzQxl0a+CvBLmm/3tmGdqcTkjg4hK4M+luXRggOnNget+A8JpDPBYEpk 2SGaCPLlTYOOJq+uM2lMPG2CF8OMQAyBA0gGrTfFXmQmmLYhEWvqFTrDp4cddFHh T4A5YRMtYVUTel1fBS2xG/8QP9tQZkYRqmj0LtkhEs5CE4u0N05laWwgU21pdGgg KGh0dHA6Ly93d3cub3Blbi5hYy51aykgPG4uc21pdGhAb3Blbi5hYy51az6JAj4E EwECACgFAlPj7DICGwMFCQWjmoAGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJ EJPB2e07PgbqdBQP/1uV92mqk0ZQ2gInT/VkLpec2JSN6DLixyAQFJBWvbTGLDim Ry6ZZgQK+bjhHholBXPaypI7K/f4W+XQ5PjHwixaPQaFI9y1Pqtrho4j2dc0gRzO UCrFZMigig8gPCarA98947WXldltSl62GkpcrwDsxCd6+cuMjfmMOSnbxYLHulUr NSCVhMIP5+ZtOncy1p2eaxq3q9PTWGtwBlkZZp/ToybWpzRudQYOI3a/Y/RcR4zY c+2DYxkA7N6WiLV3AyEcPpj4bpzc6Vay5X5s4AkcrCDry2zq/NTG583qFpEWf3gt Sm3tvRK2Aqz9xSx1F3OkY9aCpbrVvpUDB6n4D31FUII9mn24lMgjBtKW+l/f/cDR AKNQ3r9qGELvANp/VDJ1n+nKpoCKzOr8tbizTUKJL5Z+3N2IpJgqoYPNQwVLDH55 mAnUboGjjkJNZXj5IVol4+utQVN69fd+8vLd1Db2wCeeyjjttwAydYLr13A0S9LS bQgLRBtGxicH+xi/O7mdcVkf9gq3au5f597AMQ2JkVuo9DagM1CMcC+V3wyiCqWm eI62r08+ns/hl6WDNNq9C/WMV0xIl0CC6We7Hls4VKMZxyMFOpponttdB+y0IR5y f75prHdMEbtKrCTjQdyH4fQgfCbZfvy73lz+ZMKzuk5caZ/aQ/VYUqC+76xMtDpO ZWlsIFNtaXRoIChodHRwOi8vd3d3Lm9wZW4uYWMudWspIDxuZWlsLnNtaXRoQG9w ZW4uYWMudWs+iQI+BBMBAgAoBQJT4+xLAhsDBQkFo5qABgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgAAKCRCTwdntOz4G6uYrD/4+3U1FLKuqgoxCFzvNYye875H/osgo KzgH0naGoDBnU3FrUgmFH90qmcjUWd37NuNjQ9QSMX9Qr3OlUekJE3uz5f9RTTMG tSFdzt09JlCZKZ9vkykId5HpOk/LqzYJJKORr7/Nk3PaqSjJgd91DJ/lLBNXjDeY 2DU+AUGIZkFl7A/y5GktI7oaBOqqB13JVKL8Czt4TNtTw4TGL9K444Nv36aYOi5U vtJW8NTyXYJ36Xm9IcYQ7SI1hnK6mOe2KkmgSu+WYE2SgJN/k6r+8wGytkIxQBiM o9o8PomHsjXLZJb9NEFbZDmVQtV/YEEzfen1MKleeLs3CgNZvoMAujkZqb1VM4VT hol3BagLuAXzMFm30YaSZBrHBNlMuqXW+qH02idNxoqq0UX/iQf/YnXvuhS4k4EL aCEJZUTXYGAbD8QPqI48AlwcSmYG8NsnMgatETZ1pBgP9ehCmZ3137P1VOOD7iXk r42ty1ektlE05oe5xC2yDFSxunEoIXrYQRzo0yKcAXY9jKmvgtz/Ij+Hr2lz+eLA I4wIKYt6LrTQB68+VXMcFMLs6UlPRVDsomq7F31eeklL+Yz7QZJY07dMdfV+raWj JQwPoF8h187f+SnIE9/kShrG+D0hdiiOfTq+lNL9mraI3rrfTuIQ6CRQv88M7pSy FhfL/30FQzx/HLkCDQRT47T7ARAAqhei9g1G5yO0E4ZUfQwhFG9MoP2FZUIXDZYy EwPTfzpJcw5y2LG6fHPKw2VbOA4ViQiPqqB0eGSQRYUF+cPa+bxaj4SoUgqQbHNs SWf3gwB6EWrO2xy/tiiM9qBGULsonEN4d6guqo2RCR9azSkupDdOWIxqunRjOdRH BzfR5hMgfVvtN2QOLqmi0tQW5w8dSoxsNvi49eH4siUQrJNsc4syLjK5P1xBERGv oAp9THYbaLpc/dXPriVPrM9q389U6lvxM5DEgYftcqhuK/+O+6tX9O2tn9+nqSqh Vze8Ym2TeUBLxR5dbHRdaPQop6nbmqfawjI7y/JD49X+4AUbtHJ/86RvSYKWH44w N08B4c2K2AxfbgNZ+wwtEjUCPnxne8nV4nqv0mtNsxUSt1R6GBerOIptq2v+jz0e dokISXEbpduqCjZ3qlTJnGkOOtRew5iAsQlI7VeZoyKeGJpPpW//tLJIk3HhJx8P cm0Ges0LFDKDrcCOwNyXdgWLXQpEviR/DjjJlSLyJEErk9TKC8kXbeAXW1x/X5O7 Dd8Cgy1DQXJoGlMd/GgV79+5EqAEVuc8rJ6o7dCfDin24NCZ9p5K8+Fd0WxAvVVz 1hwFzLYHhg1DV3QKcYc1r8gul4Qb0RVV/zwJOQnSLTQ/UZZ6iGVs1O9mQkGbO6qR gKqr700AEQEAAYkCJQQYAQIADwUCU+O0+wIbDAUJBaOagAAKCRCTwdntOz4G6rE3 D/9LgjDv5hqZ0bmzFyWXq1HnzujE3mwggbwd1TfHIzNqE3pC4tt5uWSyLVsWVJgk NWvfBq3RD56F8xCD6uoIeMy1YP+M1v6QJ4yXwGOjft9Gea8ieZVSVb3c4vEcHc5K Q5ziw3V8IIAF8Pui2OIKia6N58dn/+ohTPbMGx2Ux2erHuW7Q2NUdXJL+J3mKYJU ASvMBpYevcHxkfvcXXySmZPPuhGqpM3TanViRQaWCpwX0GS3flvxpwHFm6h2j9gF kKxFtN2L5jdZ7dnWo1wUlwlCtHegzDHqRYGixjeBosHqDEwIsetjRpL/9cPYZKUO i/z5Hbu/ou/z/UR3F50TdSNi9x3dv3lynRgsFwhw/S4Mdh/c7quZoYkEZZ72PHhR p7yt122xecQfEHjC/9afPCloS3lPk20XOWqqleWxupkwqxHj2LAOSbAv3yv0qAtY 1GtDo5GwSV3xhJqHoP1RJ1nGN+41RisG1cQ9CgfC2bjxF+WkWS4A9SUUDKcKnYnj xrAJyw9O4auuuql/y/A0CyhxiCbapt4r6xEaOSXo/3leHV2es5KCiG7imPFIxYUL 3zhpJLWfsQQSG2+X4CA8YCPtMOcTqdDkh6ms1bho0iepoB9A0u+gKdNcI/5ZzvNq kXlw9cOooPrRWDq+kxKgf8ZvnZXHd997ok+azHIIG4dcaLkEDQRT487pEBAAjIsZ AEh9cQAkerfLW+RFo15PIcG9TRnWZvS4afQztKssp+BY+gR1JSAgdf4CldwviW40 sgGbwxoD5K6XSWdGLEZ6l4ASPutegh1W5Hf0qs6zxqkNGoTCp1JMWESxTYPk+/UF Cn5aLgXgGQOZB9TgP7gjZRoHen3lKqE/jNkcwYMZy9rJRDPYo6OVLMVlzX8U5XQx /RDL7l9xEf2MDJekb2Lf/oEiHdZw3CoFHjV9X7TuR9pYfBlDohZqDaqrjuLQsvo/ dK4xZaxe4nOypjT9EhVoo/gYgDkjvc8iU3o/jcJSqNvVxY+jrZSexO2g0xp+G+As 3iGcOzJT5n/+H0md5OPPhRQz7n6w5ID7Ciaio6PbJcVoSW039/2x/GjvEM7RqAf6 Dj21z8soGCJyr2GPZKQw0UuJepflmFkj/tKWEFrTAyqpgwX6VJJKpNTj10MLz7sM yL59/7NxFFflYI2DpH1RGIRdgsIHfFAxz6cSNTwZpPh1CL7/eZ5ocmQjWav8IO3d XloJM4eCcmfSaDyKk0JfShomx5ZZy77vIro6AB5KuI0JZfw/atFRwHZB9QAEVMu/ l5v6387U1Wd7qhpplJ+FuvqMCe4Vha3R8UqxbZMR452nj3WlsgP1iw2vcygDoW5y LWzoNiu5cpbMfNBUaj7VEybjgKK9SYblnuCffxcAAwUP/0Bn4B7qE3/egWjFApPl KlrZcjb7V5F8CccC0b8PGnmYwdKBtVeC2pTQ72kkyRHRJGo98KkmOG+5Du5HXRdz 8TLsk6/2KcVvH4/07pPKz9l4+9e98T++bjKyih8pyqdYADeUBZ664Pv4OOTsIvsx +SDkQPtWBzrnEHFaTSHXEZ42CK90XYUCgt9n9D4T5a2LrVIthXdeDSbR0FODkN8F U0/m21/Dqs8bzKCkprpSlJ20ZPs5pRhR+8Fy16q9khR86bvrs8+BOJLZnURDGEud WDLCvYi+ZasXwmHEZyiI9Lwas7CNLhNM+qRf7e3OH1lVO+YuZp0l5QqHP273etIV UnegyTcPUrACy8VdW21Jmda8f5KvoK/ASxNH9lX7f1hU3oJBQUEE5YOaG0FhhzgQ ZcKeai7qV2MkX8TyXHld1L0oUcauNi/WvjCB94HkEgM2d+rqbv7dSAUk1d/s7zfS DHEAUImoHPbFWbpSN5Z4tY8A5FQECJKAnlg2e93EXwXjBwqYgslQGiOJH0rCxqam s34YYDGw0prS/zcAWUlYtRo74ByZ0J6tBWSz8dHdvAr1Zq/GBj9U6Dvq/QFLACrn 8RxIHoWPA6G6rVUq7ZTHiqiamdv3l/ODtEMFVJkiPdBNcsHC77UBsbCWCeA47qqv WL7W/729jDUVbZ24N+lD/ghgiQIlBBgBAgAPBQJT487pAhsMBQkJZgGAAAoJEJPB 2e07PgbqbY8P/R8HfMOy09HfRtxBShnvpLCXWwdLVNTAFvltAyeuETEhgROLqtOs obVg77lz4IFv60v/dA+bkoziLV3moB/+uxWyFrUXbhuk2Quv84gAvgsNrbWyY22k SXwD9CsTMFHB2r9j0t5Sqp3Zm9iXG1YS5P4HYUAuEaAmyqIpGWPFpJnDsr6wKvn2 Gv51j0D0FIEo/wAVEmc+iAl0nzsYc+NdtqwDxOZXv51c994Qe54iCzQbBwkzwI1o uS1gGquNddtxD2qDxR8YWxGVw48XCGlYnG6if+50OdSzfudnQz2wRVIl8tBgjvLW tWRounZ43kpCwbFQrNl1AggQZ+rxHkrJmz3rOqi0b5IqbY/PyrIsvh9xZfoLy71K ldEEq5e0wo9RJ5g+wtqIzp85JcfiIYgtuJLTr8JiU9jvNK0Urc+kRqXJ/NCovQQT DIyPh146Zp1NnmZ2CbOWl3qQoCMDSeR4IhY/hBFOjM9fmcp+CONTWCyrOLOhAkfF 9C5mker7leokQ15btMBxXCvXe1q8aD57+QDAaiWxQIr5gOsqsPSMv4e3Nyva+6zi ZeNpQdeqBADn+tcteGaqXS6XRFNWPEUuKdKFrHrJ9AtNevsGkpUjwIL3oxwxU58b AmeYUhNRt8+zqzC4JJyqVDMTK5Hb3aNSngFOtU36n+92hLuFN205puNT =5k0W -----END PGP PUBLIC KEY BLOCK----- </pre> cacff542eea6d7fbc6f5b930b2c383214f20cbde 0 1593 2696 2612 2014-09-24T20:10:01Z Neil 2 /* Common tasks */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create it in <code>gitolite-admin</code> (and push the changes), then clone it on the client machine. * To add Github as a remote for a repo, git remote add github git@github.com:njae/foo.git git push github master ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 2571a7394cb398c7a36d06b5db1b1689fd218183 2697 2696 2014-09-24T20:10:20Z Neil 2 /* Common tasks */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create it in <code>gitolite-admin</code> (and push the changes), then clone it on the client machine. * To add Github as a remote for a repo, git remote add github git@github.com:neilnjae/foo.git git push github master ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] c34c4cc659de2e09bc5cfbb291d790229799bf4e 0 1541 2698 2412 2014-10-12T14:12:14Z Neil 2 /* Static IP numbers */ Added bit for wifi wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/resolv.conf</code> to include domain domain.tld search domain.tld :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 * To give a static IP to a WiFi connection using WPA Supplicant, use this interface stanza: iface wlan0 inet static address 192.168.1.101 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 wpa-essid MyWifiID wpa-psk my-secret-key # wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf # iface default inet dhcp * Change <code>/etc/resolv.conf</code> to nameserver 192.168.1.1 (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] b0e665f6be643aff0314eed530e7f5a5a6c945fc 0 1643 2699 2014-10-20T22:51:12Z Neil 2 Created page with "[https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read b…" wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. ==Timeline== {| |- valign="top" | 27 March 2014 || I notice that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restore the affected files from my own backup, but SpiderOak deletes them again. I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still plan to when the problem is resolved. |- valign="top" | 18 April 2014 || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak _again_ asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response. |} ==Conclusion== This whole saga took over four months to play out. The problem was not fixed. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seem to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. 27bd482dbd8798a13020cc9b3469f2250da66c7d 2700 2699 2014-10-20T22:57:01Z Neil 2 /* Timeline */ wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. ==Timeline== {| |- valign="top" | 27 March 2014 || I notice that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restore the affected files from my own backup, but SpiderOak deletes them again. I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still plan to when the problem is resolved. |- valign="top" | 18 April 2014 || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak _again_ asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response.<br>At this point, I gave up. |} ==Conclusion== This whole saga took over four months to play out. The problem was not fixed. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seem to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. 0fce1cc64004745992740a0f3480073ebe3b2fb0 2701 2700 2014-10-20T23:00:19Z Neil 2 wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. On 27 March 2014, I noticed that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restore the affected files from my own backup, but SpiderOak deletes them again. I contacted SpiderOak support, expecting a speedy resolution to my problem. I didn't get it. ==Timeline== {| |- valign="top" | 27 March 2014 || I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still plan to when the problem is resolved. |- valign="top" | 18 April 2014<br>(Nearly one month after the initial report) || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak _again_ asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response.<br>At this point, I gave up. |} ==Conclusion== This whole saga took over four months to play out. The problem was not fixed. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seem to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. d7a1d42285ebf17a3569eca7a8523abc017577e2 2702 2701 2014-10-20T23:01:11Z Neil 2 /* Timeline */ wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. On 27 March 2014, I noticed that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restore the affected files from my own backup, but SpiderOak deletes them again. I contacted SpiderOak support, expecting a speedy resolution to my problem. I didn't get it. ==Timeline== {| |- valign="top" | 27 March 2014 || I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still intend to do so when the problem is resolved. |- valign="top" | 18 April 2014<br>(Nearly one month after the initial report) || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak _again_ asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response.<br>At this point, I gave up. |} ==Conclusion== This whole saga took over four months to play out. The problem was not fixed. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seem to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. 642389088e1ceff1e2046c80e1acc5ff17552bae 2703 2702 2014-10-20T23:02:40Z Neil 2 /* Timeline */ wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. On 27 March 2014, I noticed that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restore the affected files from my own backup, but SpiderOak deletes them again. I contacted SpiderOak support, expecting a speedy resolution to my problem. I didn't get it. ==Timeline== {| |- valign="top" | 27 March 2014 || I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still intend to do so when the problem is resolved. |- valign="top" | 18 April 2014<br>(Nearly one month after the initial report) || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak ''again'' asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response.<br>At this point, I gave up. |} ==Conclusion== This whole saga took over four months to play out. The problem was not fixed. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seem to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. f31a1042b36112887ce11f313e5ce0bab7740b5b 2704 2703 2014-10-20T23:06:54Z Neil 2 /* Conclusion */ wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. On 27 March 2014, I noticed that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restore the affected files from my own backup, but SpiderOak deletes them again. I contacted SpiderOak support, expecting a speedy resolution to my problem. I didn't get it. ==Timeline== {| |- valign="top" | 27 March 2014 || I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still intend to do so when the problem is resolved. |- valign="top" | 18 April 2014<br>(Nearly one month after the initial report) || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak ''again'' asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response.<br>At this point, I gave up. |} ==Conclusion== This whole saga took over four months to play out. The problem was not fixed. SpiderOak continues to delete arbitrary files from its backup, and does so silently. This is not good behaviour from a service that depends on its reliability and trustworthiness for its use. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seems to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. I'm a satisfied customer of theirs. 70b3cefda66d354f05469b5f65d368f7e8aa289b 2705 2704 2014-10-21T07:55:34Z Neil 2 wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. On 27 March 2014, I noticed that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restore the affected files from my own backup, SpiderOak deletes them again. I contacted SpiderOak support, expecting a speedy resolution to my problem. I didn't get it. ==Timeline== {| |- valign="top" | 27 March 2014 || I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still intend to do so when the problem is resolved. |- valign="top" | 18 April 2014<br>(Nearly one month after the initial report) || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak ''again'' asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response.<br>At this point, I gave up. |} ==Conclusion== This whole saga took over five months to play out. The problem was not fixed. SpiderOak continues to delete arbitrary files from its backup, and does so silently. This is not good behaviour from a service that depends on its reliability and trustworthiness for its use. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seems to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. I'm a satisfied customer of theirs. 55489ff3108d9b33dc3174a3eafce293089b71a3 2706 2705 2014-10-21T10:27:15Z Neil 2 wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. On 27 March 2014, I noticed that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restored the affected files from my own backup, SpiderOak deleted them again. I contacted SpiderOak support, expecting a speedy resolution to my problem. I didn't get it. I have to consider both SpiderOak and [https://encryptr.org/ Encryptr] (based on SpiderOak's engine) as unreliable places to store data. I recommend you don't use either. [https://wuala.com Wuala] seems to be better. ==Timeline== {| |- valign="top" | 27 March 2014 || I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still intend to do so when the problem is resolved. |- valign="top" | 18 April 2014<br>(Nearly one month after the initial report) || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak ''again'' asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days while he investigates further. |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response.At this point, I gave up. |} At this point, I gave up. ==Conclusion== This whole saga took over five months to play out. The problem was not fixed. SpiderOak continues to delete arbitrary files from its backup, and does so silently. This is not good behaviour from a service that depends on its reliability and trustworthiness for its use. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seems to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. If you're using SpiderOak for backups, I suggest you check that all your files still contain what you thought they did. It's not enough to just check whether the file exists: you have to check that the file contains the data it should. The [https://encryptr.org/ Encryptr] "secure" password locker is based on SpiderOak's technology. Unfortunately, that means I also consider it unreliable. I no longer use SpiderOak and I recommend you don't either. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. I'm a satisfied customer of theirs. 85cc8e93f8c4f97299a340c5121d18d42daf50f5 0 1 2707 2689 2014-10-21T10:43:18Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == {| align="right" {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://ello.co/neilnjae @neilnjae] on Ello * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] |} Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://wuala.com Wuala] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 182a06225110c3cd23742f8e26d2629c5781e3be 2711 2707 2014-12-12T11:42:42Z Neil 2 /* Neil */ Added Keybase.io link wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == {| align="right" {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://ello.co/neilnjae @neilnjae] on Ello * [https://keybase.io/neilnjae NeilNjae] on Keybase.io * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] |} Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://wuala.com Wuala] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. c0219709d6bd5716d59e98ea2e36649689b704a5 2715 2711 2015-08-20T07:56:46Z Neil 2 Added BTC address wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == {| align="right" {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://ello.co/neilnjae @neilnjae] on Ello * [https://keybase.io/neilnjae NeilNjae] on Keybase.io * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] * Send me BitCoins at [https://blockchain.info/address/1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV 1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV] |} Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://wuala.com Wuala] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. a79a26fc4ad1566921ac363fa0a393611a4348bd 2717 2715 2015-08-25T10:18:55Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == <div style="float: right;"> {| {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://ello.co/neilnjae @neilnjae] on Ello * [https://keybase.io/neilnjae NeilNjae] on Keybase.io * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] * Send me BitCoins at [https://blockchain.info/address/1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV 1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV] |} </div> Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://wuala.com Wuala] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 8529f07436cc9942b7c74f24f29a858d47b22c73 2727 2717 2015-11-04T10:52:48Z Neil 2 /* Neil */ wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == <div style="float: right;"> {| {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://ello.co/neilnjae @neilnjae] on Ello * [https://keybase.io/neilnjae NeilNjae] on Keybase.io * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] * Send me BitCoins at [https://blockchain.info/address/1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV 1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV] |} </div> Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://tresorit.com/ Tresorit] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 1181c5c8fe61eff114f17b1bb52ceb1cf4f3f08a 0 1643 2708 2706 2014-10-22T19:47:05Z Neil 2 /* Timeline */ wikitext text/x-wiki [https://www.spideroak.com SpiderOak] seems like a good idea: online backup of your files, synchronised across your devices, with the data stored securely and unable to be read by SpiderOak employees and government. Unfortunately, it's no good if the backup service deletes your files behind your back. I was a user of their free service for a while. I'd over doubled my free storage through referrals to friends and family. It was about time to pay over some money for a useful service. I was just about to take advantage of their offer of unlimited storage for US$125 p.a. when the problems started. On 27 March 2014, I noticed that several files in my synced folder now have sizes of zero bytes. The files themselves are present, with the same names and permissions, just no content. 139 files are affected across the several directories I synchronise. It's not clear when this problem occurred, but I think it was about 12 March. When I restored the affected files from my own backup, SpiderOak deleted them again. I contacted SpiderOak support, expecting a speedy resolution to my problem. I didn't get it. I have to consider both SpiderOak and [https://encryptr.org/ Encryptr] (based on SpiderOak's engine) as unreliable places to store data. I recommend you don't use either. [https://wuala.com Wuala] seems to be better. ==Timeline== {| |- valign="top" | 27 March 2014 || I report the problem to SpiderOak and receive their automated acknowledgement. |- valign="top" | 28 March 2014 || [Person A] from SpiderOak customer relations contacts me and asks for copies of basic logs. I send them over. |- valign="top" | 29 March 2014 || [Engineer B] from SpiderOak contacts me to acknowledge receipt of the logs and says they'll contact me soon with an update. |- valign="top" | 2 April 2014 || [Engineer B] says that there will be a delay in addressing my problem and credits me with some additional storage. I say that I was planning to move to the paid plan when the problem started and still intend to do so when the problem is resolved. |- valign="top" | 18 April 2014<br>(Nearly one month after the initial report) || [Engineer B] responds and credits me with some additional storage. |- valign="top" | 17 May 2014<br>(Nearly two months after the initial report) || I contact [Engineer B]. He apologies for the delay in addressing my problem and says it's been now a priority and now passed to [Senior Engineer C]. [Engineer B] credits me with more storage. I hear nothing from [Senior Engineer C]. |- valign="top | 23 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top | 29 May 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 9 June 2014 || I'm now fed up with the lack of response from SpiderOak. I make a public post on their blog and other social media, again explaining that I'm willing to pay for their service if it's reliable. [Person D] from customer relations and [Engineer E] contact me independently to express sympathy with my frustration. |- valign="top" | 11 June 2014 || [Engineer B] contacts me and says that [Senior Engineer C] requires more information and asks me for additional logs. I send them over. |- valign="top" | 13 June 2014 || [Senior Engineer C] contacts me to acknowledge receipt of the additional logs and says it will take him a while to go through it. |- valign="top" | 20 June 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 27 June 2014<br>(Three months after I raised the ticket) || I contact SpiderOak ''again'' asking for an update. This time I cc the two founders.<br>Three hours later, [Senior Engineer C] contacts me with his suspicions of the cause of the problem (it may be related to a bug in the Windows client). He offers me a new account with SpiderOak (the first time this is offered as a fix).<br>I offer to do whichever helps SpiderOak get to the bottom of the problem. |- valign="top" | 30 June 2014 || [Senior Engineer C] asks me to hold on for an additional four days while he investigates further. |- valign="top | 8 July 2014 || [Senior Engineer C] asks me to wait for another couple of days, citing delays caused by US holiays. "I'm trying to have a conclusion on your ticket very soon," he says |- valign="top" | 13 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 17 July 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 19 July 2014 || [Senior Engineer C] apologies for the lack of progress. |- valign="top" | 25 July 2014 || [Engineer B] contacts me and offers to set up a new account. I accept. |- valign="top" | 30 July 2014<br>(Over four months since the original report) || [Engineer B] tells me the new account is ready. However, the new account does not have even my original space allocation, let alone the bonus storage I've gained throughout this problem "resolution." This means the new account isn't useful. |- valign="top" | 1 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 6 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 8 August 2014 || I contact SpiderOak asking for my storage to be allocated to the new account. I get no response. |- valign="top" | 11 August 2014 || My new account is finally credited with my storage. I add my files to it. SpiderOak immediately deletes some of them again. I raise a new ticket with SpiderOak and add a comment to the old one. I collect the basic and additional logs and send them in. |- valign="top" | 16 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 20 August 2014 || I contact SpiderOak asking for a progress report. I get no response. |- valign="top" | 2 September 2014 || I contact SpiderOak asking for a progress report. I send a copy to SpiderOak's founders. I get no response. |} At this point, I gave up. ==Conclusion== This whole saga took over five months to play out. The problem was not fixed. SpiderOak continues to delete arbitrary files from its backup, and does so silently. This is not good behaviour from a service that depends on its reliability and trustworthiness for its use. If my problem was related to a bug in the Windows client, it seems that bug still exists. SpiderOak seems to have no interest in helping non-business customers, or in converting their enthusiastic users into paying customers. If you're using SpiderOak for backups, I suggest you check that all your files still contain what you thought they did. It's not enough to just check whether the file exists: you have to check that the file contains the data it should. The [https://encryptr.org/ Encryptr] "secure" password locker is based on SpiderOak's technology. Unfortunately, that means I also consider it unreliable. I no longer use SpiderOak and I recommend you don't either. I'm now paying [https://wuala.com Wuala] for a secure backup and synchronisation service. I've had a couple of questions for them and have had very quick and full responses from them. I'm a satisfied customer of theirs. 7e79e9f40057a503e8b58f0e0319ba03c3826a40 0 1532 2709 2499 2014-11-20T17:25:45Z Neil 2 /* Initial Dovecot configuration */ Included how to disable the login listener on port 993 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] a138175fff96c610f0f00340e7578c5c5cdb3d6d 2716 2709 2015-08-25T08:23:05Z Neil 2 /* Initial Dovecot configuration */ Add inbox namespace to 10-mail wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.domain.tld:993 <verbiage snipped> * OK Dovecot ready. :and test the login as before. Mail clients using IMAPS should connect to <code>imap.domain.tld:993</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] e3f80121f5951d9969461bd578dd53f4ef0ea33f 0 1593 2710 2697 2014-12-12T11:37:23Z Neil 2 /* Common tasks */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create its entry in <code>gitolite-admin/conf/gitolite.conf</code> and push the changes. This will create an empty repo on the server. Then clone it on the client machine, ignoring the message about cloning an empty repo. * To add Github as a remote for a repo, git remote add github git@github.com:neilnjae/foo.git git push github master ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] d547fe95fc9d384e3a5b276ed1c65369b7bb0e0c 2712 2710 2014-12-12T11:47:59Z Neil 2 Added section on changing the template wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf.d/gitweb</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Reload apache root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Changing the default template== Modify the standard template if you want changes to affect all new repos created. Do this, for instance, if you want to use [https://keybase.io/neilnjae Keybase.io] to automatically sign all commits by adding the file <code>hooks/pre-commit</code>: #!/bin/sh # Run Keybase to sign the commit keybase dir sign -p git git add SIGNED.md ===Server side=== Modify the contents of <code>/usr/share/git-core/templates/</code> as you see fit, such as adding the commit hook above. This should affect all repos created with Gitolite. ===Client side=== Copy <code>/usr/share/git-core/templates/</code> to somewhere local, such as <code>~/.git_template</code>. Modify the contents as you see fit. Tell Git to use this template rather than the default one: user@desktop:~$ git config --global init.templatedir '~/.git_template' This should affect all new repos created on the client, outside of Gitolote. ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create its entry in <code>gitolite-admin/conf/gitolite.conf</code> and push the changes. This will create an empty repo on the server. Then clone it on the client machine, ignoring the message about cloning an empty repo. * To add Github as a remote for a repo, git remote add github git@github.com:neilnjae/foo.git git push github master ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 740e52dffde8666719cf99f3f1219c11d9425540 2721 2712 2015-08-27T17:03:18Z Neil 2 /* Setup gitweb */ Updated for apache conf enabling wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf-available/gitweb.conf</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options +FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Enable the gitweb config and reload Apache: root@server:~# a2enconf gitweb root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Changing the default template== Modify the standard template if you want changes to affect all new repos created. Do this, for instance, if you want to use [https://keybase.io/neilnjae Keybase.io] to automatically sign all commits by adding the file <code>hooks/pre-commit</code>: #!/bin/sh # Run Keybase to sign the commit keybase dir sign -p git git add SIGNED.md ===Server side=== Modify the contents of <code>/usr/share/git-core/templates/</code> as you see fit, such as adding the commit hook above. This should affect all repos created with Gitolite. ===Client side=== Copy <code>/usr/share/git-core/templates/</code> to somewhere local, such as <code>~/.git_template</code>. Modify the contents as you see fit. Tell Git to use this template rather than the default one: user@desktop:~$ git config --global init.templatedir '~/.git_template' This should affect all new repos created on the client, outside of Gitolote. ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create its entry in <code>gitolite-admin/conf/gitolite.conf</code> and push the changes. This will create an empty repo on the server. Then clone it on the client machine, ignoring the message about cloning an empty repo. * To add Github as a remote for a repo, git remote add github git@github.com:neilnjae/foo.git git push github master ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] ac91f6142a89d5548c7fa3927b7739c352297c5f 0 1526 2713 2695 2015-04-17T08:22:27Z Neil 2 /* Enable the Compose key */ Added additional compose sequences wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. [[Category:Server setup]] 2ab225bf8b404508d48f0ca1fbb00aaca13ba42f 2722 2713 2015-10-30T14:52:38Z Neil 2 Added Calibre server section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/etc/systemd/system/calibre-server.service</code> [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --with-library /home/user/path/to/calibre-library --auto-reload --port 8080 [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl daemon-reload root@desktop:~# service calibre-server start * Check again that the Calibre server is running. [[Category:Server setup]] ffe2c29500e4dbe714e45beb5031b3c8eeb8c331 0 1630 2714 2687 2015-06-09T07:58:10Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity build-essential calibre couchdb-bin create-resources curl dos2unix exfat-utils fbreader finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gnome-games gnubg gnuplot-qt # Possible conflict with gnuplot-nox gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve hal-info haskell-mode haskell-platform haskell-platform-prof llvm ibus inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs npm normalize-audio octave ogmtools openshot p7zip pandoc pavucontrol pavumeter pdftk pitivi postgresql postgresql-client r-base sbcl scratch screen scribus scribus-template seahorse shotwell slime sqlite3 stellarium swi-prolog tesseract-ocr texlive-full texlive-bibtex-extra texlive-pictures texlive-science tk-dev tk tofrodos traceroute units vino virtuoso-server vlc whois wireshark xine-plugin xine-ui xinetd xtightvncviewer zenity Packages which were either already installed by default, or no longer necessary. # biblatex # blueman # cheese # cmake # cython # dpkg-dev # easytag ? # eog # ffmpeg # fgetty # fop # gmusicbrowser # gucharmap # guvcview # html2text # idle # idle3 # paprefs # pastebinit # perlmagick # python-all # python3-all # r-recommended # samba # sni-qt # software-center # software-properties-gtk # sound-juicer # thunderbird # tuxpaint # tuxpaint-config # xsane 1d1d1adfd9ae5c9425cd81249c005dc769058ba1 0 1534 2718 2463 2015-08-25T11:16:26Z Neil 2 /* Creating the farm */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/load.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/resources/ * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### # If PHP's memory limit is very low, some operations may fail. #ini_set( 'memory_limit', '20M' ); ini_set( 'memory_limit', '128M' ); ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs # $wgArticlePath = "$wgScript?title=$1"; $wgArticlePath = "/$1"; ### $wgUsePathInfo = true; ### $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "/mediawiki/images/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 8a0b2891c63adc459c59438985a11cbfb219136c 2719 2718 2015-08-25T11:17:00Z Neil 2 /* Rewriting paths */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == root@server:~# apt-get install mediawiki mediawiki-extensions mediawiki-math == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/load.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/resources/ * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### # If PHP's memory limit is very low, some operations may fail. #ini_set( 'memory_limit', '20M' ); ini_set( 'memory_limit', '128M' ); ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs # $wgArticlePath = "$wgScript?title=$1"; $wgArticlePath = "/$1"; ### $wgUsePathInfo = true; ### $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "/mediawiki/images/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] a6e18a6f27ccb60345ae458ec7b6f0e01113dcd0 0 1542 2720 2497 2015-08-25T15:52:19Z Neil 2 Added avahi section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. ==Allow network printing== Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. ===Update Avahi demon=== The Avahi demon may not allow printing to networked printers, as the client machines may not understand the ''hostname''.local naming convention that CUPS expects. On all machines, add <code>myhostname</code> to the end of the <code>hosts:</code> line: hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4 myhostname Restart Avahi and CUPS: root@server:~# service avahi-daemon restart root@server:~# service cups restart == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it, as described on the [[Scanner setup]] page. ==Bugfix== CUPS on Ubuntu 12.04 has a [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/872483 bug that affects the USB backend], preventing the printer printing more than one job with it needing resetting. Once the printer is set up, issue the command: root@server:~# lpadmin -p <printer> -o usb-no-reattach-default=true == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] 5662eaecb72a19a267d7865e9d3502e85b4d6b4c 0 1543 2723 2669 2015-10-30T18:16:44Z Neil 2 /* Set up ufw */ Added KDE Connect rule wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. ==Server== === Remove ufw === The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. === Add a bespoke iptables firewall === * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> ==Desktop== The routing tables on the router mean that people can't get direct access to the desktop machine. In this case, <code>ufw</code> is sufficient. ===Set up ufw=== Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.2.0/24 to any port 1714:1764 (The last rule is for [https://community.kde.org/KDEConnect KDE Connect]). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] f2f225bde2709794a42b1694efac09b4f934c374 2724 2723 2015-10-30T18:16:57Z Neil 2 /* Set up ufw */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. ==Server== === Remove ufw === The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. === Add a bespoke iptables firewall === * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> ==Desktop== The routing tables on the router mean that people can't get direct access to the desktop machine. In this case, <code>ufw</code> is sufficient. ===Set up ufw=== Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.2.0/24 to any port 1714:1764 (The last rule is for [https://community.kde.org/KDEConnect KDE Connect]). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 5f516aa31036278c771f3435e8c925c77f5d001e 2725 2724 2015-10-31T13:35:12Z Neil 2 /* Set up ufw */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. ==Server== === Remove ufw === The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. === Add a bespoke iptables firewall === * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> ==Desktop== The routing tables on the router mean that people can't get direct access to the desktop machine. In this case, <code>ufw</code> is sufficient. ===Set up ufw=== Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 (The last two rules are for [https://community.kde.org/KDEConnect KDE Connect] and [https://github.com/MatejVancik/amaroKontrol amaroKontrol] respectively). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] a9d93209e884916fbbf033ff07597c2214f777d1 2726 2725 2015-10-31T13:38:06Z Neil 2 /* Check ufw's status */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. ==Server== === Remove ufw === The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. === Add a bespoke iptables firewall === * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> ==Desktop== The routing tables on the router mean that people can't get direct access to the desktop machine. In this case, <code>ufw</code> is sufficient. ===Set up ufw=== Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 (The last two rules are for [https://community.kde.org/KDEConnect KDE Connect] and [https://github.com/MatejVancik/amaroKontrol amaroKontrol] respectively). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 1714:1764/tcp ALLOW IN 192.168.1.0/24 8484/tcp ALLOW IN 192.168.1.0/24 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 30d670110c6065bfd3ab96efdbbf65ecc2105432 0 1594 2728 2479 2015-11-17T15:45:22Z Neil 2 /* AWStats configuration */ Added default domain wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ == AWStats configuration == ==SiteDomain== AWStats doesn't like not knowing a default domain, even though the domain is specified in each site-specific configuration below. Modify <code>/etc/awstats/awstats.conf</code> to include a default domain, such as the one that serves the AWStats pages. SiteDomain="site.domain.tld" === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# service apache2 reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 5879001603ea83494c20c7aaf343b46765e4acd9 0 1531 2729 2502 2015-11-17T15:56:02Z Neil 2 /* Adding TLS */ Updated TLS security level directive wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 9bd29e9b1109a167ef02bf9eec455741c0508a77 2730 2729 2015-11-17T16:03:10Z Neil 2 /* Open submission port (587) */ Fewer overrides needed wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking the headers of incoming emails and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 84e86c8181b0ea53f89f395b41026db9fca4b3d5 2731 2730 2015-11-18T13:16:57Z Neil 2 /* Block addresses that are known to spammers */ Added recipient_checks instructions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] fe7293cc9ba0d1b085936df2aa905093ab6e2f9b 2732 2731 2015-11-18T13:17:43Z Neil 2 /* Configure Postfix as a backup MX server for other domains */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 5ffe960e8966b6809293dac4d4fe04e78ec6c197 2733 2732 2015-11-18T13:17:55Z Neil 2 /* Configure Postfix as a backup MX server for other domains */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 61bdfc31ed13a396072dd4733e9ccf95bfb7ccc4 2734 2733 2015-11-18T13:18:37Z Neil 2 /* Block addresses that are known to spammers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] ae12e8b4392130572259387cd9677540203ac6a3 2735 2734 2015-11-18T13:19:11Z Neil 2 /* Allowing off-site users with SASL */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] b7280299a2cd42a35028622068a016b162b71298 2736 2735 2015-11-18T13:20:05Z Neil 2 /* Block addresses that are known to spammers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] f1a3a7822ca3fde115ba62fb6f2eb455e9fe55c5 2737 2736 2015-11-18T13:20:41Z Neil 2 /* Block addresses that are known to spammers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] c89e9aa32b7cc6bc08e08a5398d76c977e1dac38 2738 2737 2015-11-18T13:21:58Z Neil 2 /* Basic installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * Modify <code>/etc/postfix/main.cf</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks permit_sasl_authenticated permit_mynetworks permit_mx_backup reject_unauth_destination permit_mx_backup_networks = otherdomain1.com otherdomain2.com * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 34c43967823fd88c0f30f838db958bb23e51dd98 2739 2738 2015-11-18T14:18:23Z Neil 2 /* Configure Postfix as a backup MX server for other domains */ Updated to use relay_domains wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 28294a78ebe8010b279ac0c6625d2487f35e067a 2740 2739 2015-11-20T10:29:58Z Neil 2 Added SPF section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== ===SPF=== * Add the following as a <code>TXT</code> record to your DNS entry. v=spf1 mx a mx:backup.com :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF. The mail log should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] fd2cc3ea3fb6f8f1090ff40fb2cfbd67534a52b5 2741 2740 2015-11-20T10:38:32Z Neil 2 /* Authenication and verification */ Added blurb and SPF tests wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. * Add the following as a <code>TXT</code> record to your DNS entry. v=spf1 mx a mx:backup.com :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 85604dad4eeeefb689084bccde79d712501f6dad 2742 2741 2015-11-20T10:59:12Z Neil 2 /* Authenication and verification */ Added stub sections wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. * Add the following as a <code>TXT</code> record to your DNS entry. v=spf1 mx a mx:backup.com :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ==DKIM== To add: see documents at https://help.ubuntu.com/community/Postfix/DKIM , https://rtcamp.com/tutorials/mail/dkim-postfix-ubuntu/ ==DANE== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 3c9bdc3655397d033dca01d521cfcac1af1daf60 2743 2742 2015-11-20T10:59:31Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. * Add the following as a <code>TXT</code> record to your DNS entry. v=spf1 mx a mx:backup.com :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== To add: see documents at https://help.ubuntu.com/community/Postfix/DKIM , https://rtcamp.com/tutorials/mail/dkim-postfix-ubuntu/ ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] ba298f2f0a214771f3529ba8c409c01d87e41d5e 2744 2743 2015-11-20T11:22:25Z Neil 2 /* SPF */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== To add: see documents at https://help.ubuntu.com/community/Postfix/DKIM , https://rtcamp.com/tutorials/mail/dkim-postfix-ubuntu/ ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 0c3d572f771f19cad26f58f5301af033695fb51b 2745 2744 2015-11-20T22:59:13Z Neil 2 /* SPF */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== To add: see documents at https://help.ubuntu.com/community/Postfix/DKIM , https://rtcamp.com/tutorials/mail/dkim-postfix-ubuntu/ ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 20440798f06d8609fc6451ed13eac1ceef544efb 2746 2745 2015-11-20T23:15:39Z Neil 2 /* DKIM */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== To add: see documents at https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim , https://help.ubuntu.com/community/Postfix/DKIM , https://rtcamp.com/tutorials/mail/dkim-postfix-ubuntu/ , http://www.stevejenkins.com/blog/2015/03/installing-opendmarc-rpm-via-yum-with-postfix-or-sendmail-for-rhel-centos-fedora/ ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 5a7b8bfa28a7096a58b8d389dfa7616799a8cbeb 2747 2746 2015-11-20T23:17:42Z Neil 2 /* SPF */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== To add: see documents at https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim , https://help.ubuntu.com/community/Postfix/DKIM , https://rtcamp.com/tutorials/mail/dkim-postfix-ubuntu/ , http://www.stevejenkins.com/blog/2015/03/installing-opendmarc-rpm-via-yum-with-postfix-or-sendmail-for-rhel-centos-fedora/ ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] c217c94b33095107a64ef75e90d0587c84e79e2a 2748 2747 2015-11-21T13:11:20Z Neil 2 /* DKIM */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] e05f7aa4452fce5e05a5966d3eb3e41d9747629f 2749 2748 2015-11-21T13:44:43Z Neil 2 /* Authenication and verification */ Added start of DMARC wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 09ce4acde6d44935fd899c6a6d26e2b0c14d81a7 2750 2749 2015-11-30T13:04:52Z Neil 2 /* SPF */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] da3f0ec76df72795cf06efcd5ac292a6ccfad6bc 2751 2750 2015-11-30T14:47:57Z Neil 2 /* DMARC */ wikitext 2752 2751 2015-11-30T14:48:20Z Neil 2 /* DMARC */ wikitext 2753 2752 2015-11-30T14:49:57Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing incomoing mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/8 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Logging incoming mail''' The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f cat /dev/null > ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@njae.me.uk/ neil@njae.me.uk :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] ba038cd73d11274b2c4c44ea352e70d1d8059074 2754 2753 2015-11-30T15:04:02Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/8 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f cat /dev/null > ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@njae.me.uk/ neil@njae.me.uk :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] b57a0252bf2aca4d77c00ebb6a41659367c951b4 2755 2754 2015-11-30T15:04:54Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/8 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f cat /dev/null > ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 3a632a4f724429a24a5cd4343613dce9761d701e 2756 2755 2015-12-01T15:22:18Z Neil 2 /* DMARC */ Added note on versions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/8 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f cat /dev/null > ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 44a88e7eaad967c861669324593da77cc1ab9e1d 0 1531 2757 2756 2015-12-01T15:23:33Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f cat /dev/null > ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 892a4eb7dcc14f1c9e5f8b8ccc4386a32632bb7a 2758 2757 2015-12-01T15:29:38Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/ssl/domain/certs/mail-cert.pem smtpd_tls_key_file=/etc/ssl/domain/private/mail-key.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] a8b46dd17cac3a7f13684c4c6c53a46f8b779fdd 2766 2758 2016-01-19T13:37:13Z Neil 2 /* Adding TLS */ Updated for LetsEncrypt certificates wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 1ec5671f1f0727b9240c33478c144b40daae4224 2767 2766 2016-01-19T13:37:22Z Neil 2 /* Adding TLS */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] f6d2adf91c1ac7d3e70c75fb767bf07b54dc338d 2778 2767 2016-02-03T09:49:45Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/header_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] f0be30e344f451f7f0c45ad1a234d163f774013f 2784 2778 2016-03-23T10:34:51Z Neil 2 /* Block addresses that are known to spammers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] f502cadbcf8bbf0b1a16f2aef5b402e739c53c12 2805 2784 2016-12-31T14:52:42Z Neil 2 /* Amavis + ClamAV + SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] a8bb0c7e6f6d02e5c6e3c7c5f7a2def1b40c5007 2806 2805 2016-12-31T14:54:12Z Neil 2 /* Amavis + ClamAV + SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lha nomarch pax rar unrar unzip zip zoo unzoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] ae50b043305c9fa9613dc0733dc1b5a3680baf02 0 1644 2759 2015-12-29T13:20:33Z Neil 2 Created page with "This is not an Open University page." wikitext text/x-wiki This is not an Open University page. 97ff5ca3d4eaee6327c33cb50780a23e1f2d76b0 2760 2759 2015-12-29T13:25:23Z Neil 2 wikitext text/x-wiki This is not an Open University page, but it could look like one where you enter your login details. 640d6c1fbacb57fb12248b2d91e05d64adcfe1b8 0 1526 2761 2722 2016-01-06T15:11:43Z Neil 2 /* Create systemd job */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~# ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. * Log into <code>user@desktop.domain.tld</code> * Append the <code>id_rsa.pub</code> key to <code>someone@server</code>'s <code>~/.ssh/authorized_keys</code> file: user@desktop:~# cat ~/.ssh/id_rsa.pub | ssh someone@server.domain.tld 'cat >> /home/someone/.ssh/authorized_keys' <code>user@desktop</code> should now be able to start SSH sessions as <code>someone@server</code> without giving <code>someone</code>'s password: user@desktop:~# ssh someone@server <login banner snipped> someone@server:~# Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --with-library /home/user/path/to/calibre-library --auto-reload --port 8080 [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# ln -s /lib/systemd/system/calibre-server.service /etc/systemd/system/calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# service calibre-server start * Check again that the Calibre server is running. [[Category:Server setup]] 64f093d07df3fb2b929511f7f3d63f638756ffbf 2781 2761 2016-02-13T17:47:00Z Neil 2 /* Password-less SSH logins */ Redone using ssh-copy-id wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --with-library /home/user/path/to/calibre-library --auto-reload --port 8080 [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# ln -s /lib/systemd/system/calibre-server.service /etc/systemd/system/calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# service calibre-server start * Check again that the Calibre server is running. [[Category:Server setup]] 915535987dda281a755ca159a5bd4d290cad0a72 2783 2781 2016-03-08T09:31:02Z Neil 2 /* Create systemd job */ Updated instructions to start the systemd job wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --with-library /home/user/path/to/calibre-library --auto-reload --port 8080 [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service <s>root@desktop:~# ln -s /lib/systemd/system/calibre-server.service /etc/systemd/system/calibre-server.service</s> root@desktop:~# systemctl daemon-reload <s>root@desktop:~# service calibre-server start</s> root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. [[Category:Server setup]] f200d80f7b80a6357b493a28022d7b16bb8c4c3d 2792 2783 2016-06-14T11:05:59Z Neil 2 Added how to change tooltip colours wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --with-library /home/user/path/to/calibre-library --auto-reload --port 8080 [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service <s>root@desktop:~# ln -s /lib/systemd/system/calibre-server.service /etc/systemd/system/calibre-server.service</s> root@desktop:~# systemctl daemon-reload <s>root@desktop:~# service calibre-server start</s> root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] e3f96c870f169d541f6b744b22e2b82e417d3902 0 1533 2762 2606 2016-01-11T16:56:34Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# service apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <cocde>/etc/apache2/ssl/certs</code>. root@server:~# mkdir -p /etc/apache2/ssl/private root@server:~# mkdir -p /etc/apache2/ssl/certs root@server:~# cp /etc/ssl/njae/certs/squirrelmail-cert.pem /etc/apache2/ssl/certs/ root@server:~# cp /etc/ssl/njae/private/squirrelmail-key.insecure.pem /etc/apache2/ssl/private/ Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. root@server:~# chown -R www-data:www-data /etc/apache2/ssl/ root@server:~# chmod -R 640 /etc/apache2/ssl/ Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/private/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off * To protect against the Poodle attack, disable SSL v3. Edit <code>/etc/apache2/mods-enabled/ssl.conf</code> so the <code>SSLProtocol</code> line reads: SSLProtocol all -SSLv3 -SSLv2 * Disable the weak RC4 algorithm, and prioritise better algorithms. Edit <code>/etc/apache2/mods-enabled/ssl.conf</code> SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:EC\ DHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!\ aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 After any of these changes, reload the Apache configuration. root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 59bca32e5af9bf84c460f2b08ea21ff0f82a84c3 2764 2762 2016-01-19T13:25:17Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# service apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <cocde>/etc/apache2/ssl/certs</code>. root@server:~# mkdir -p /etc/apache2/ssl/private root@server:~# mkdir -p /etc/apache2/ssl/certs root@server:~# cp /etc/ssl/njae/certs/squirrelmail-cert.pem /etc/apache2/ssl/certs/ root@server:~# cp /etc/ssl/njae/private/squirrelmail-key.insecure.pem /etc/apache2/ssl/private/ Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. root@server:~# chown -R www-data:www-data /etc/apache2/ssl/ root@server:~# chmod -R 640 /etc/apache2/ssl/ Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/private/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ===Note to self=== On Apache version 2.4.8 and above, replace: SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem with: SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off * To protect against the Poodle attack, disable SSL v3. Edit <code>/etc/apache2/mods-enabled/ssl.conf</code> so the <code>SSLProtocol</code> line reads: SSLProtocol all -SSLv3 -SSLv2 * Disable the weak RC4 algorithm, and prioritise better algorithms. Edit <code>/etc/apache2/mods-enabled/ssl.conf</code> SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:EC\ DHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!\ aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 After any of these changes, reload the Apache configuration. root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 83f5e0e2e47f53115597e2c3cf2b964d77fcc73b 2773 2764 2016-01-19T14:05:00Z Neil 2 /* Secure HTTP */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# service apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Copy the SSL certificate and insecure key file [[SSL certificate generation|generated earlier]] to <code>/etc/apache2/ssl/certs</code>. root@server:~# mkdir -p /etc/apache2/ssl/private root@server:~# mkdir -p /etc/apache2/ssl/certs root@server:~# cp /etc/ssl/njae/certs/squirrelmail-cert.pem /etc/apache2/ssl/certs/ root@server:~# cp /etc/ssl/njae/private/squirrelmail-key.insecure.pem /etc/apache2/ssl/private/ Make sure that this directory is not visible to the world, as that could compromise the security of the SSL traffic if the certificate is read by anyone else. root@server:~# chown -R www-data:www-data /etc/apache2/ssl/ root@server:~# chmod -R 640 /etc/apache2/ssl/ Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateFile /etc/apache2/ssl/certs/squirrelmail-cert.pem SSLCertificateKeyFile /etc/apache2/ssl/private/squirrelmail-key.insecure.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ===Note to self=== On Apache version 2.4.8 and above, replace: SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem with: SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off * To protect against the Poodle attack, disable SSL v3. Edit <code>/etc/apache2/mods-enabled/ssl.conf</code> so the <code>SSLProtocol</code> line reads: SSLProtocol all -SSLv3 -SSLv2 * Disable the weak RC4 algorithm, and prioritise better algorithms. Edit <code>/etc/apache2/mods-enabled/ssl.conf</code> SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:EC\ DHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!\ aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 After any of these changes, reload the Apache configuration. root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 7c41938380bbe93c969e0415879265c01bf6ece7 2774 2773 2016-01-19T14:07:47Z Neil 2 /* Secure HTTP */ Updated for Let's Encrypt wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# service apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/cert.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ===Note to self=== On Apache version 2.4.8 and above, replace: SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem with: SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off * To protect against the Poodle attack, disable SSL v3. Edit <code>/etc/apache2/mods-enabled/ssl.conf</code> so the <code>SSLProtocol</code> line reads: SSLProtocol all -SSLv3 -SSLv2 * Disable the weak RC4 algorithm, and prioritise better algorithms. Edit <code>/etc/apache2/mods-enabled/ssl.conf</code> SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:EC\ DHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!\ aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4 After any of these changes, reload the Apache configuration. root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 713abfa124fae630fd0e4512a63ba6c7caa51026 0 1532 2763 2716 2016-01-19T13:17:51Z Neil 2 /* Testing Dovecot */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/ssl/domain/certs/imap-cert.pem ssl_key = </etc/ssl/domain/private/imap-key.pem * Include these lines at the end of <code>/etc/dovecot.conf</code> : # This file contains the SSL key !include_try /etc/ssl/domain/private/imap-key-password * Create the file to contain the key passcode: root@server:~# echo 'ssl_key_password = mykey' > /etc/ssl/domain/private/imap-key-password root@server:~# chown root:root /etc/ssl/domain/private/imap-key-password root@server:~# chmod 0400 /etc/ssl/domain/private/imap-key-password :(The key is the same one you gave when creating the [[SSL certificate generation|SSL certificates]].) == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 32c7dd9308e5086d008bb019413c47fb31193a7d 2765 2763 2016-01-19T13:35:55Z Neil 2 /* Initial Dovecot configuration */ Updated for letsencrypy certificates wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] fc93f6646e27459570421ad48bc20bdff345ad43 2800 2765 2016-12-30T19:52:22Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap ==== Note to self: don't disable to protocol, turn them off in 10-master.conf ==== * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /home/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. ==== Note to self: also disable pop3, as below: ==== service pop3-login { inet_listener pop3 { #port = 110 port = 0 } inet_listener pop3s { #port = 995 #ssl = yes port = 0 } } * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] c09161ce165ca2b836ba922516d0bc589f3bc1c3 2801 2800 2016-12-30T19:57:19Z Neil 2 /* Note to self: don't disable to protocol, turn them off in 10-master.conf */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap ==== Note to self: don't disable to protocol, turn them off in 10-master.conf ==== * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. ==== Note to self: also disable pop3, as below: ==== service pop3-login { inet_listener pop3 { #port = 110 port = 0 } inet_listener pop3s { #port = 995 #ssl = yes port = 0 } } * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 95a6eb8c1af54cd4b62e5d88e17470c546d2cadf 2802 2801 2016-12-31T12:25:44Z Neil 2 /* Create virtual users */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap ==== Note to self: don't disable to protocol, turn them off in 10-master.conf ==== * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. ==== Note to self: also disable pop3, as below: ==== service pop3-login { inet_listener pop3 { #port = 110 port = 0 } inet_listener pop3s { #port = 995 #ssl = yes port = 0 } } * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth test user@domain.tld mailpassword root@server:~# doveadm auth login user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] ea5115d4ae78c3db7131349ef2c764dab8638bb3 2803 2802 2016-12-31T13:22:28Z Neil 2 /* Note to self: also disable pop3, as below: */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.squirrelmail.org/ Squirrelmail] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap ==== Note to self: don't disable to protocol, turn them off in 10-master.conf ==== * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. ==== Note to self: also disable pop3, as below: ==== service pop3-login { inet_listener pop3 { #port = 110 port = 0 } inet_listener pop3s { #port = 995 #ssl = yes port = 0 } } * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl = yes ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth test user@domain.tld mailpassword root@server:~# doveadm auth login user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 077c74560328164a39e9b248798ad8953eda8280 0 1530 2768 2667 2016-01-19T13:42:58Z Neil 2 Neil moved page [[SSL certificate generation]] to [[Self-signed SSL certificate generation]]: Obsolete wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} [http://www.eclectica.ca/ Eclectica] has a good [http://www.eclectica.ca/howto/ssl-cert-howto.php guide to setting up SSL certificates]. Basically, I followed the instructions given there, except that I didn't rebuild the <tt>/etc/ssl/openssl.cnf</tt> file, as described at Eclectica. == Initial configuration == * Install OpenSSL, installed as part of the SSH server package: root@server:~# apt-get install openssh-server * I did all the creation work in <code>/etc/ssl/domain</code>, so create the directory structure: root@server:~# cd /etc/ssl root@server:/etc/ssl# mkdir -p domain/private domain/certs domain/newcerts * Create a couple of files for the database of keys root@server:~# echo '01' > domain/serial root@server:~# touch domain/index.txt * Create a list of key revocations root@server:~# echo '01' > domain/crlnumber * Create the <code>/etc/ssl/openssl.cnf</code> file as shown below. == Create the root certifiate == root@server:/etc/ssl# openssl req -new -x509 -extensions v3_ca \ -keyout domain/private/cakey.pem -out domain/cacert.pem \ -days 3650 -config ./openssl.cnf Using configuration from ./openssl.cnf Generating a 1024 bit RSA private key .......++++++ ..........................++++++ writing new private key to 'domain/private/cakey.pem' Enter PEM pass phrase:demo Verifying password - Enter PEM pass phrase:demo ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Organization Name (company) [The Sample Company]:<enter> Organizational Unit Name (department, division) []:CA Division Email Address []:ca@sample.com Locality Name (city, district) [Your Town]:<enter> State or Province Name (full name) [Your State]:<enter> Country Name (2 letter code) [UK]:<enter> Common Name (hostname, IP, or your name) []:My Root CA * Carefully note the passphrase used to generate this root CA. This generates a private key in <tt>domain/private/cakey.pem</tt> and a root CA certificate in <tt>domain/cacert.pem</tt> . It is the latter that can be distributed. == Create a certificate signing request == Now to create a certificate signing request for all of the systems that need them. For the server, that's the MTA, the IMAP server, and the Squirrelmail web mail server. root@server:/etc/ssl# openssl req -new -nodes \ -out domain/squirrelmail-req.pem \ -keyout domain/private/squirrelmail-key.pem -config ./openssl.cnf ... Organizational Unit Name (department, division) []:Webmail Server Email Address []:postmaster@sample.com Common Name (hostname, IP, or your name) []:squirrelmail.domain.tld ... Change the name <tt>squirrelmail-req.pem</tt> for the purpose needed. Make sure that the common name given is the fully-qualified domain name of the machine that will use the certificate. Each CSR will create a pair of files: a private key <tt>squirrelmail-key.pem</tt> and a signing request <tt>squirrelmail-req.pem</tt> == Sign the request == root@server:/etc/ssl# openssl ca -out domain/certs/squirrelmail-cert.pem \ -config ./openssl.cnf -infiles domain/squirrelmail-req.pem Using configuration from ./openssl.cnf Enter PEM pass phrase:demo Check that the request matches the signature Signature ok The Subjects Distinguished Name is as follows organizationName :PRINTABLE:'The Sample Company' organizationalUnitName:PRINTABLE:'Webmail Server' emailAddress :IA5STRING:'postmaster@sample.com' localityName :PRINTABLE:'Your Town' stateOrProvinceName :PRINTABLE:'Your State' countryName :PRINTABLE:'UK' commonName :PRINTABLE:'squirrelmail.domain.tld' Certificate is to be certified until Dec 8 04:37:38 2002 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated This creates the signed certificate in <tt>squrrelmail-cert.pem</tt> and a copy in <tt>newcerts/<serial>.pem</tt>. == Installing certificates == This is detailed in the information for the particular applications. One wrinkle is that the certificate for Apache needs to be modified so that Apache doesn't request the passphrase every time it starts. root@server:/etc/ssl# openssl rsa -in domain/private/squirrelmail-key.pem \ -out domain/private/squirrelmail-key.insecure.pem Full details on setting up Apache are in the [[Web server setup]] notes. Read more on [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html generating SSL keys for Apache] and [http://heinous.org/wiki/Apache_SSL_Notes more notes for Apache + SSL]. ==Revoking a certificate== When certificates expire or become compromised, they can be revoked. * Revoke a certificate with the command: root@server:/etc/ssl# openssl ca -revoke domain/certs/squirrelmail-cert.pem * Once you've done all the revocations you need to, generate the certificate revocation list (CRL): root@server:/etc/ssl# openssl ca -gencrl -out domain/root.crl * You can check the contents of the revocation list with: root@server:/etc/ssl# openssl ca -gencrl -out domain/root.crl * Copy the new CRL to the correct directory on the wweb server: root@server:/etc/ssl# cp domain/root.crl /var/www/domain.tld/root.crl You can then generate new certificates to replace the revoked ones. ==Viewing the contents of a certificate== root@server:/etc/ssl# openssl x509 -in domain/certs/squirrelmail-cert.pem -text == Contents of <tt>/etc/ssl/openssl.cnf</tt> == Note the change to the <tt>dir</tt> entry in the <tt>[ CA_default ]</tt> section. # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # HOME = . RANDFILE = $ENV::HOME/.rnd oid_section = new_oids #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./domain # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/root.crl # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options crl_extensions = crl_ext default_days = 3650 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = sha1 # which md to use. preserve = no # keep passed DN ordering policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 1024 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert string_mask = nombstr req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = UK countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Your State localityName = Locality Name (eg, city) localityName_default = Your Town 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Your Org Name organizationalUnitName = Organizational Unit Name (eg, section) commonName = Common Name (eg, YOUR name) commonName_default = Domain commonName_max = 64 emailAddress = Email Address emailAddress_default = webmaster@domain.tld emailAddress_max = 64 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. basicConstraints=CA:FALSE nsComment = "OpenSSL Generated Certificate" subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer keyUsage = digitalSignature,keyEncipherment extendedKeyUsage = serverAuth crlDistributionPoints = URI:http://domain.tld/root.crl subjectAltName = @alt_names [alt_names] DNS.1 = domain.tld DNS.2 = *.domain.tld [ v3_req ] basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always basicConstraints = CA:true [ crl_ext ] authorityKeyIdentifier=keyid:always,issuer:always [ proxy_cert_ext ] basicConstraints=CA:FALSE nsComment = "OpenSSL Generated Certificate" subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo [[Category:Server setup]] a191db3db5734baab2de5103fa179218d047cf09 0 1645 2769 2016-01-19T13:42:58Z Neil 2 Neil moved page [[SSL certificate generation]] to [[Self-signed SSL certificate generation]]: Obsolete wikitext text/x-wiki #REDIRECT [[Self-signed SSL certificate generation]] 0b3b8b96081138542ad21433a676cf7d5215583a 2770 2769 2016-01-19T13:44:56Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 6c3be10a9f353ecadf37c32a9249c9dc7a4d1ca7 2771 2770 2016-01-19T13:59:35Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} I ''think'' this requires an existing webserver already listening for HTTPS connections. ==Get certificates from Lets Encrypt== Get the certificates from: root@server:# /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring \ -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring \ -d domain.tld -d www.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring \ -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring \ -d work.domain.tld root@server:# service apache2 reload Note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Automatically update certificates== Create the file <code>/etc/cron.daily/renew-certificates</code>: #!/bin/bash /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d domain.tld -d www.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d work.domain.tld service apache2 reload Make it executable: root@server:# chmod a+x /etc/cron.daily/renew-certificates ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 785ed376b3692443028bb089f735440a8955567a 2772 2771 2016-01-19T14:03:54Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} I ''think'' this requires an existing webserver already listening for HTTPS connections. You may need to bootstrap an [[Web server setup|Apache installation]] with a [[Self-signed SSL certificate generation|self-signed certificate]]. ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.org/en/latest/using.html#installation install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== Get the certificates from: root@server:# /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring \ -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring \ -d domain.tld -d www.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring \ -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring \ -d work.domain.tld root@server:# service apache2 reload Note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Automatically update certificates== Create the file <code>/etc/cron.daily/renew-certificates</code>: #!/bin/bash /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d domain.tld -d www.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d work.domain.tld service apache2 reload Make it executable: root@server:# chmod a+x /etc/cron.daily/renew-certificates ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 07d3fd04e331c5b069293672e897198fae9a7ef4 2775 2772 2016-01-21T12:51:55Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.org/en/latest/using.html#installation install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# /opt/letsencrypt/letsencrypt-auto -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d domain.tld -d www.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d work.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Automatically update certificates== Create the file <code>/etc/cron.daily/renew-certificates</code>: #!/bin/bash /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d domain.tld -d www.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d work.domain.tld service apache2 reload Make it executable: root@server:# chmod a+x /etc/cron.daily/renew-certificates ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 905a22c2d728a36a9f49922fee9afe8221887fa7 2782 2775 2016-02-22T11:06:16Z Neil 2 Added bits on expanding and viewing certificates wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.org/en/latest/using.html#installation install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# /opt/letsencrypt/letsencrypt-auto -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d domain.tld -d www.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d work.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--expand</code> option and list all the domains for the certificate: root@server:~# /opt/letsencrypt/letsencrypt-auto -d domain.tld -d www.domain.tld -d other.domain.tld ==Check the contents of a certificate== If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text root@server:~# service apache2 reload Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==Automatically update certificates== Create the file <code>/etc/cron.daily/renew-certificates</code>: #!/bin/bash /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d domain.tld -d www.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --keep-until-expiring -d work.domain.tld service apache2 reload Make it executable: root@server:# chmod a+x /etc/cron.daily/renew-certificates ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 7943d8bc38c878468c1a70c173b0b6b88e0220b6 2785 2782 2016-04-23T16:30:36Z Neil 2 /* Automatically update certificates */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.org/en/latest/using.html#installation install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# /opt/letsencrypt/letsencrypt-auto -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d domain.tld -d www.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld root@server:# /opt/letsencrypt/letsencrypt-auto -d work.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--expand</code> option and list all the domains for the certificate: root@server:~# /opt/letsencrypt/letsencrypt-auto -d domain.tld -d www.domain.tld -d other.domain.tld ==Check the contents of a certificate== If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text root@server:~# service apache2 reload Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==Automatically update certificates== Create the file <code>/etc/cron.daily/renew-certificates</code>: #!/bin/bash /opt/letsencrypt/letsencrypt-auto certonly --apache --text --non-interactive --keep-until-expiring -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --non-interactive --keep-until-expiring -d domain.tld -d www.domain.tld /opt/letsencrypt/letsencrypt-auto certonly --apache --text --non-interactive --keep-until-expiring -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld service apache2 reload Make it executable: root@server:# chmod a+x /etc/cron.daily/renew-certificates ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 5baf8af0a5dc2759731608f010e0c0c82e5b7838 0 1646 2776 2016-02-02T09:33:25Z Neil 2 Created page with "{|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup..." wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |align="center" width="50%"| [[MySQL configuration|MySQL config]] |} The [[Postfix server setup|Postfix]] installation will handle receiving mail and will allow MUAs to connect, we need an extra bit to allow automated mail originating from other machines to be delivered to Postfix. [https://wiki.debian.org/sSMTP sSMTP] does just that. * Install sSMTP: root@desktop:~# aptitude install ssmtp * Edit <code>/etc/ssmtp/ssmtp.conf</code> to these non-comment lines: root=root@domain.tld mailhub=server.domain.tld hostname=desktop.domain.tld rewriteDomain=domain.tld :Note that the machine names should be the hostnames directly, not aliases set up by [[DNS server setup|DNS]]. * Test the setup by attempting to send a message: user@desktop:~$ echo "this is the body" | mail -s "Subject" root@domain.tld ==See also== * [https://wiki.archlinux.org/index.php/SSMTP sSMTP on the Arch Linux wiki] * [https://wiki.debian.org/sSMTP sSMTP on the Debian Linux wiki] [[Category:Server setup]] 18f705ec98d905f272b5a6f296ef8067096938c9 0 1545 2777 2604 2016-02-03T09:49:40Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Desktop mail forwarding]] |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server mysql-client == Passwords == The first thing is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the anonymous user accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR <nowiki>''</nowiki>@'host_name' = PASSWORD('newpwd'); :where <code>host_name</code> is the name of this host, e.g. <code>server</code>. :(seemingly not needed for Ubuntu 10.04) * Add passwords to the <code>root</code> accounts root@server:~# mysql -u root mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'127.0.0.1' = PASSWORD('newpwd'); == Remote access == If you want to access MySQL databases from a remote machine, you'll need to have MySQL accept connections across the network. Edit <code>/etc/mysql/my.cnf</code> and change the <code>bind-address</code> entry to be the IP of the machine: bind-address = 192.168.2.252 The restart MySQL: root@server:~# /etc/init.d/mysql restart Note that the [[Firewall setup|firewall]] is configured to only allow connections to MySQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] [[Category:Server setup]] 5b753eed354f6e56a2cd2a3be71b3dfcacc16206 0 1515 2779 2607 2016-02-03T09:50:07Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 12.04 LTS and 13.10 (adapted from 8.04 LTS by way of 10.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Squirrelmail)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Desktop mail forwarding]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 5ec279c1572a9c68475cc7ed1d30ad40c9f4116e 0 1588 2780 2342 2016-02-11T09:02:09Z Neil 2 Added Fail2Ban section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Firewall setup|Firewall]] |align="center" width="50%"| [[DNS server setup|DNS server]] |} If you open port 22 (for SSH access) to the outside world, you'll want to protect it against opportunistic, possibly hostile, login attempts by unauthorised people. There are three tactics you should consider: # Ensure that all accounts on the SSH server are protected by strong passwords; # Restrict the IP addresses that can access the SSH port; # Change the SSH port to something non-standard. The first item is outside the scope of this page. Note that not all accounts on your computer will allow logins anyway. Those that do will require strong passwords. == Changes to config file == But before that, there are a couple of changes that need to be made to the SSH server's <code>/etc/ssh/sshd_config</code> file. Disable root logins by changing the line to this # Authentication: PermitRootLogin no to prevent attempts to log in as root. Add the line AllowUsers user1 user2 user3 to only allow SSH access to these users. Then restart the SSH server: root@server:~# /etc/init.d/ssh restart == Changing the SSH port == If you're using a different port, use one above port 1024 as this is as high as most port scanners routinely go. There are three ways of doing this: # Change the port sshd listens on # Use the router to map a non-standard external port to port 22 on the machine on the LAN # Use the machine's firewall to map a non-standard external port to port 22 on the machine on the LAN The advantage of the first method is that it doesn't require fiddling with port mappings in firewalls, but it will require the extra port to be opened (and port 22 closed) and all SSH access to that machine to use the new port. The advantage of the other two methods is that you can keep the machine listening on port 22 for connections within your LAN. === Change the port sshd listens on === Change the port specified in <code>/etc/ssh/sshd_config</code>: # Run ssh on a non-standard port: Port 1111 (If you want <code>sshd</code> to listen on several ports, give several <code>Port</code> lines.) To connect to this machine, either specify the new port on the command line (<code>ssh -p 1111 user@machine</code>) or specify the port to use in the client user's <code>~/.ssh/config</code> file: # Client ~/.ssh/config Host myserver HostName 72.232.194.162 User bob Port 1111 === Use the router to map a non-standard external port === Use the router's 'virtual servers' or 'services' option. Open the higher port and ensure it maps to port 22 on the target machine. Not all routers can do this port mapping. === Use the machine's firewall to map a non-standard external port === Open the higher port on the router and have it map to the same port on the target machine. Then, modify the prerouting table in the <code>iptables</code> rule set to map the incoming port to port 22, as used for SSH. Do this by including this fragment in the <code>/etc/iptables.rules</code> file, listed below. ## ------------------------------------------------------------ # DNAT mappings # # These rules transform the packet destinations of incoming packets. # For instance, make packets coming to port 1111 be sent to port 22 (for SSH) # Change the port that you can use for SSH. # Connections to port 1111 will be mapped to port 22, where # the SSH daemon will hear them. iptables -t nat -A PREROUTING -i $IFACE -p tcp --dport 1111 -j DNAT --to ${IPADDR}:22 == Filter connection attempts in the firewall == If you only connect from a few machines, you can restrict connections to those machines using the SSH server's firewall. Modify <code>/etc/iptables.rules</code> to mention only the IP numbers you want: ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --syn -s $LAN --dport ssh -m limit --limit 2/m --limit-burst 10 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ssh -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --syn -s 123.123.123.123 --dport ssh -m limit --limit 1/m --limit-burst 3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s 123.123.123.123 --dport ssh -j ACCEPT The first and third rules limit the number of connection attempts that will be allowed. The second and fourth rules restrict other packets to being from the specified IP addresses. ==Automatic IP blocking== The final line of defence is to dynamically block IP addresses that are probing SSH (and other) ports. [http://www.fail2ban.org/wiki/index.php/Main_Page Fail2Ban] does this, by watching logs for repeated failed login attempts, then modifying the host's firewall to reject connections from that IP. * Install Fail2Ban: root@server:~# aptitude install fail2ban * Copy <code>/etc/fail2ban/jail.conf</code> to <code>/etc/fail2ban/jail.local</code>, then modify <code>/etc/fail2ban/jail.local</code>: ignoreip = 127.0.0.1/8 192.168.1.0/24 :This line prevents Fail2Ban triggering on failed attempts from this network. * Restart Fail2Ban: root@server:~# service fail2ban restart The remaining default settings are sensible: three failed attempts in ten minutes (600 seconds) results in that IP being banned for ten minutes. If you want to change them, adjust the <code>findtime</code>, <code>maxretry</code>, and <code>bantime</code> settings in <code>/etc/fail2ban/jail.local</code>. == See also == * [http://wiki.centos.org/HowTos/Network/SecuringSSH CentOS guide to securing SSH] * [https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-ubuntu-14-04 Digital Ocean's guide to Fail2Ban] * [[Firewall setup]] page [[Category:Server setup]] eab24ad7fffe550712b47d694203e937a8fa52c8 0 1540 2786 2430 2016-05-25T08:58:03Z Neil 2 /* Create users */ Added PSQL backup user wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'password'; CREATE ROLE postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 6c31740279faf3c7d8061167424d229b38596918 2787 2786 2016-05-25T09:03:08Z Neil 2 /* Create users */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'password'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 3f613b3e2398a53baae8d00d44248e0b33003247 2788 2787 2016-05-25T09:18:06Z Neil 2 /* Create users */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/home/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 3 days old nice find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +2 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/vmail/*' --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /home/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in information_schema mysql site1wikidb site2wikidb; do nice mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 600 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done # Create Subversion backups nice svnadmin dump /opt/svn/repo1 | bzip2 > "$BACKUPFILENAME.svn-repo1.bz2" nice svnadmin dump /opt/svn/repo2 | bzip2 > "$BACKUPFILENAME.svn-repo2.bz2" chmod 600 "$BACKUPFILENAME.opt.tar.bz2" chmod 600 "$BACKUPFILENAME.etc.tar.bz2" chmod 600 "$BACKUPFILENAME.home.tar.bz2" chmod 600 "$BACKUPFILENAME.vmail.tar.bz2" chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" chmod 600 "$BACKUPFILENAME.www.tar.bz2" chmod 600 "$BACKUPFILENAME.cgi.tar.bz2" chmod 600 "$BACKUPFILENAME.svn-repo1.bz2" chmod 600 "$BACKUPFILENAME.svn-repo2.bz2" To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Subversion dumps, you may need to create the repository: root@server:~# svnadmin create /opt/svn/repo1 then load the repository dump file root@server:~# bunzip2 REPO_DUMP_FILE | svnadmin load /opt/svn/repo1 and finally, perhaps change file ownerships and permissions: root@server:~# chown -R www-data:subversion /opt/svn/repo1 root@server:~# chmod -R g+rws /opt/svn/repo1/db/ == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 9a0e7b4eab5b7a8134f17ab5f7ef600e65949141 2789 2788 2016-05-26T07:16:20Z Neil 2 /* Server backup */ General update wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/home/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.temujin.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore files, use root@server:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx and maybe with <code>--absolute-names</code> (to restore anchored at root) To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 7ba4f64524fdaca40b75e315b95f3c6646160cef 2790 2789 2016-05-26T07:17:08Z Neil 2 /* Desktop backup */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/home/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore files, use root@server:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx and maybe with <code>--absolute-names</code> (to restore anchored at root) To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 2be8f657421de6220c9c049573d2edbeec76a9ca 2791 2790 2016-05-26T07:17:24Z Neil 2 /* Server backup */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/home/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 6077aab26b357469863a637c25c2e14d3eab5c78 0 1593 2793 2721 2016-10-26T15:12:00Z Neil 2 /* Common tasks */ Added github import section. wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf-available/gitweb.conf</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options +FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Enable the gitweb config and reload Apache: root@server:~# a2enconf gitweb root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Changing the default template== Modify the standard template if you want changes to affect all new repos created. Do this, for instance, if you want to use [https://keybase.io/neilnjae Keybase.io] to automatically sign all commits by adding the file <code>hooks/pre-commit</code>: #!/bin/sh # Run Keybase to sign the commit keybase dir sign -p git git add SIGNED.md ===Server side=== Modify the contents of <code>/usr/share/git-core/templates/</code> as you see fit, such as adding the commit hook above. This should affect all repos created with Gitolite. ===Client side=== Copy <code>/usr/share/git-core/templates/</code> to somewhere local, such as <code>~/.git_template</code>. Modify the contents as you see fit. Tell Git to use this template rather than the default one: user@desktop:~$ git config --global init.templatedir '~/.git_template' This should affect all new repos created on the client, outside of Gitolote. ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create its entry in <code>gitolite-admin/conf/gitolite.conf</code> and push the changes. This will create an empty repo on the server. Then clone it on the client machine, ignoring the message about cloning an empty repo. * To import a repo into Github, use the Github "import repository" functionality, using the url <code><nowiki>git://git.domain.tld/repo.git</nowiki></code> * To add Github as a remote for a repo, git remote add github git@github.com:neilnjae/foo.git git push github master ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 597671d8cef3af7230d0dd1e2e736521dbec1d45 2794 2793 2016-12-18T12:02:30Z Neil 2 /* Common tasks */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf-available/gitweb.conf</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options +FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Enable the gitweb config and reload Apache: root@server:~# a2enconf gitweb root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Changing the default template== Modify the standard template if you want changes to affect all new repos created. Do this, for instance, if you want to use [https://keybase.io/neilnjae Keybase.io] to automatically sign all commits by adding the file <code>hooks/pre-commit</code>: #!/bin/sh # Run Keybase to sign the commit keybase dir sign -p git git add SIGNED.md ===Server side=== Modify the contents of <code>/usr/share/git-core/templates/</code> as you see fit, such as adding the commit hook above. This should affect all repos created with Gitolite. ===Client side=== Copy <code>/usr/share/git-core/templates/</code> to somewhere local, such as <code>~/.git_template</code>. Modify the contents as you see fit. Tell Git to use this template rather than the default one: user@desktop:~$ git config --global init.templatedir '~/.git_template' This should affect all new repos created on the client, outside of Gitolote. ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create its entry in <code>gitolite-admin/conf/gitolite.conf</code> and push the changes. This will create an empty repo on the server. Then clone it on the client machine, ignoring the message about cloning an empty repo. * To import a repo into Github, use the Github "import repository" functionality, using the url <code><nowiki>git://git.domain.tld/repo.git</nowiki></code> . Note that you may need to remove files larger than 100Mb, following [https://help.github.com/articles/remove-sensitive-data/ Github's instructions]. * To add Github as a remote for a repo, git remote add github git@github.com:neilnjae/foo.git git push github master ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 7d62b5ac58ac290b4ea4aa52ba23d3160d176908 0 1543 2795 2726 2016-12-30T11:47:34Z Neil 2 /* Set up ufw */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. ==Server== === Remove ufw === The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. === Add a bespoke iptables firewall === * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> ==Desktop== The routing tables on the router mean that people can't get direct access to the desktop machine. In this case, <code>ufw</code> is sufficient. ===Set up ufw=== Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 (The last two rules are for [https://community.kde.org/KDEConnect KDE Connect] and [https://github.com/MatejVancik/amaroKontrol amaroKontrol] respectively). Enable ufw and add rules for the services the server machine will provide: root@server:~# ufw enable root@server:~# ufw allow ssh root@server:~# ufw allow http root@server:~# ufw allow https root@server:~# ufw allow domain root@server:~# ufw allow ipp root@server:~# ufw allow imap root@server:~# ufw allow smtp root@server:~# ufw allow submission root@server:~# ufw allow from 192.168.1.0/24 to any port mysql root@server:~# ufw allow from 192.168.1.0/24 to any port postgresql root@server:~# ufw allow git root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 27017:27019 # Mongo root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 28017 # Mongo web interface ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 1714:1764/tcp ALLOW IN 192.168.1.0/24 8484/tcp ALLOW IN 192.168.1.0/24 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) == Iptables.rules == This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END == Iptables script == #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] bbd7bc74b7847ca475ecb8a8989c35d53704d162 2796 2795 2016-12-30T11:57:05Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. ==Desktop== The routing tables on the router mean that people can't get direct access to the desktop machine. In this case, <code>ufw</code> is sufficient. ==Set up ufw== Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 (The last two rules are for [https://community.kde.org/KDEConnect KDE Connect] and [https://github.com/MatejVancik/amaroKontrol amaroKontrol] respectively). Enable ufw and add rules for the services the server machine will provide: root@server:~# ufw enable root@server:~# ufw allow ssh root@server:~# ufw allow http root@server:~# ufw allow https root@server:~# ufw allow domain root@server:~# ufw allow ipp root@server:~# ufw allow imap root@server:~# ufw allow smtp root@server:~# ufw allow submission root@server:~# ufw allow from 192.168.1.0/24 to any port mysql root@server:~# ufw allow from 192.168.1.0/24 to any port postgresql root@server:~# ufw allow git root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 27017:27019 # Mongo root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 28017 # Mongo web interface ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 1714:1764/tcp ALLOW IN 192.168.1.0/24 8484/tcp ALLOW IN 192.168.1.0/24 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) ==Server (old)== === Remove ufw === The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. === Add a bespoke iptables firewall === * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> === Iptables.rules === This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END === Iptables script === #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 4b6aaa18dd7785a8bc598f9ecaf96a0454a03e05 2797 2796 2016-12-30T12:01:35Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} I used to do [[#Server (Old)|firewall setup manually]] for the server, but <code>ufw</code> has now improved to the point where I use that. ==Set up ufw== Enable ufw and add rules for the services the server machine will provide: root@server:~# ufw enable root@server:~# ufw allow ssh root@server:~# ufw allow http root@server:~# ufw allow https root@server:~# ufw allow domain root@server:~# ufw allow ipp root@server:~# ufw allow imap root@server:~# ufw allow smtp root@server:~# ufw allow submission root@server:~# ufw allow from 192.168.1.0/24 to any port mysql root@server:~# ufw allow from 192.168.1.0/24 to any port postgresql root@server:~# ufw allow git root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 27017:27019 # Mongo root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 28017 # Mongo web interface Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 (The last two rules are for [https://community.kde.org/KDEConnect KDE Connect] and [https://github.com/MatejVancik/amaroKontrol amaroKontrol] respectively). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 1714:1764/tcp ALLOW IN 192.168.1.0/24 8484/tcp ALLOW IN 192.168.1.0/24 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) ==Server (old)== The base of this script came from [http://www.sns.ias.edu/~jns/wp/2006/01/12/iptables-example-rulesets/ James Stevens's example firewall rulesets]. Adjustments to the <code>/proc</code> entries are [http://www.securityfocus.com/infocus/1711 described by Brian Hatch] at SecurityFocus. === Remove ufw === The first thing to do is to disable the default <code>ufw</code> package: root@server:~# ufw disable With that done, you can move on to installing a more flexible firewall. === Add a bespoke iptables firewall === * Add the <code>/etc/iptables.rules</code> file and the <code>/etc/iptables</code> file. Adjust <code>/etc/iptables.rules</code> to suit, depending on what services you want to offer with each machine. * Make both files executable root@server:~# chown root:root /etc/iptables.rules root@server:~# chmod +x /etc/iptables.rules root@server:~# chown root:root /etc/iptables root@server:~# chmod +x /etc/iptables * Install it in the startup sequence with root@server:~# update-rc.d iptables start 13 2 3 4 5 . stop 13 S 0 1 6 . :(note the full stops) Information on iptables from the [http://iptables-tutorial.frozentux.net/chunkyhtml/index.html iptables tutorial]. Names of the services used are listed in <code>/etc/services</code> === Iptables.rules === This is the rule set for the server machine #!/bin/bash ## Iptables ruleset # Based on an original by James Stephens (jns@ias.edu) # http://www.sns.ias.edu/~jns/ ## ============================================================ # # Load appropriate modules. modprobe ip_tables modprobe ip_conntrack modprobe ip_conntrack_ftp # These lines are here in case rules are already in place and the script is ever rerun on the fly. # We want to remove all rules and pre-exisiting user defined chains and zero the counters # before we implement new rules. iptables -F iptables -X iptables -Z # Set up a default policy for the built-in chains. # If we modify and re-run the script mid-session then (because we have a default DROP # policy), what happens is that there is a small time period when packets are denied until # the new rules are back in place. There is no period, however small, when packets we # don't want are allowed. iptables -P INPUT DROP iptables -P FORWARD DROP iptables -P OUTPUT ACCEPT ## =========================================================== ## Some definitions: IFACE="eth0" IPADDR="192.168.1.251" # this machine's address NAMESERVER_1="192.168.1.251" # desktop.domain.tld NAMESERVER_2="1.2.3.4" # ISP's primary nameserver NAMESERVER_3="1.2.3.5" # ISP's secondary nameserver ROUTER="192.168.1.1" LAN="192.168.1.0/24" BROADCAST="192.168.1.255" LOOPBACK="127.0.0.0/8" CLASS_A="10.0.0.0/8" CLASS_B="172.16.0.0/12" CLASS_C="192.168.0.0/16" CLASS_D_MULTICAST="224.0.0.0/4" CLASS_E_RESERVED_NET="240.0.0.0/5" P_PORTS="0:1023" UP_PORTS="1024:65535" TR_SRC_PORTS="32769:65535" TR_DEST_PORTS="33434:33523" ## ============================================================ ## Kernel flags # To dynamically change kernel parameters and variables on the fly you need # CONFIG_SYSCTL defined in your kernel. I would advise the following: # Disable response to ping. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all # echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all # Disable response to broadcasts. # You don't want yourself becoming a Smurf amplifier. echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts # Don't accept source routed packets. Attackers can use source routing to generate # traffic pretending to be from inside your network, but which is routed back along # the path from which it came, namely outside, so attackers can compromise your # network. Source routing is rarely used for legitimate purposes. echo "0" > /proc/sys/net/ipv4/conf/all/accept_source_route # Disable ICMP redirect acceptance. ICMP redirects can be used to alter your routing # tables, possibly to a bad end. echo "0" > /proc/sys/net/ipv4/conf/all/accept_redirects echo "0" > /proc/sys/net/ipv4/conf/all/send_redirects # Enable bad error message protection. echo "1" > /proc/sys/net/ipv4/icmp_ignore_bogus_error_responses # Turn on reverse path filtering. This helps make sure that packets use # legitimate source addresses, by automatically rejecting incoming packets # if the routing table entry for their source address doesn't match the network # interface they're arriving on. This has security advantages because it prevents # so-called IP spoofing, however it can pose problems if you use asymmetric routing # (packets from you to a host take a different path than packets from that host to you) # or if you operate a non-routing host which has several IP addresses on different # interfaces. (Note - If you turn on IP forwarding, you will also get this). for interface in /proc/sys/net/ipv4/conf/*/rp_filter; do echo "1" > ${interface} done # Log spoofed packets, source routed packets, redirect packets. echo "1" > /proc/sys/net/ipv4/conf/all/log_martians # Make sure that IP forwarding is turned off. We only want this for a multi-homed host. echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/tcp_syncookies # protect against DoS attacks ## ============================================================ # RULES ## LOOPBACK # Allow unlimited traffic on the loopback interface. iptables -A INPUT -i lo -j ACCEPT # Standard actions for logging and dropping packets ## LOGGING # You don't have to split up your logging like I do below, but I prefer to do it this way # because I can then grep for things in the logs more easily. One thing you probably want # to do is rate-limit the logging. I didn't do that here because it is probably best not too # when you first set things up ................. you actually really want to see everything going to # the logs to work out what isn't working and why. You cam implement logging with # "-m limit --limit 6/h --limit-burst 5" (or similar) before the -j LOG in each case. # Create a new table to allow easy access iptables -N log-and-drop # Any fragment not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -f -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES FRAGMENTS: " iptables -A log-and-drop -i $IFACE -f -j DROP # Any udp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p udp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES UDP-IN: " iptables -A log-and-drop -i $IFACE -p udp -j DROP # Any icmp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p icmp -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES ICMP-IN: " iptables -A log-and-drop -i $IFACE -p icmp -j DROP # Any tcp not already allowed is logged and then dropped. iptables -A log-and-drop -i $IFACE -p tcp -j LOG --log-ip-options --log-tcp-options --log-level debug --log-prefix "IPTABLES TCP-IN: " iptables -A log-and-drop -i $IFACE -p tcp -j DROP # Anything else not already allowed is logged and then dropped. # It will be dropped by the default policy anyway ........ but let's be paranoid. iptables -A log-and-drop -i $IFACE -j LOG --log-ip-options --log-level debug --log-prefix "IPTABLES PROTOCOL-X-IN: " iptables -A log-and-drop -i $IFACE -j DROP ## SYN-FLOODING PROTECTION # This rule maximises the rate of incoming connections. In order to do this we divert tcp # packets with the SYN bit set off to a user-defined chain. Up to limit-burst connections # can arrive in 1/limit seconds ..... in this case 10 connections in one second. After this, one # of the burst is regained every second and connections are allowed again. The default limit # is 3/hour. The default limit burst is 5. # (was --limit-burst 4 ) iptables -N syn-flood iptables -A INPUT -i $IFACE -p tcp --syn -j syn-flood iptables -A syn-flood -m limit --limit 5/s --limit-burst 100 -j RETURN iptables -A syn-flood -j log-and-drop ### Allow any packets related to connections we've started iptables -A INPUT -i $IFACE -m state --state ESTABLISHED,RELATED -j ACCEPT # FRAGMENTS # I have to say that fragments scare me more than anything. # Sending lots of non-first fragments was what allowed Jolt2 to effectively "drown" # Firewall-1. Fragments can be overlapped, and the subsequent interpretation of such # fragments is very OS-dependent (see this paper for details). # I am not going to trust any fragments. # Log fragments just to see if we get any, and deny them too. iptables -A INPUT -i $IFACE -f -j log-and-drop ## Make sure NEW tcp connections are SYN packets iptables -A INPUT -i $IFACE -p tcp ! --syn -m state --state NEW -j log-and-drop ## SPOOFING # Most of this anti-spoofing stuff is theoretically not really necessary with the flags we # have set in the kernel above ........... but you never know there isn't a bug somewhere in # your IP stack. # # Special case: silently drop packets broadcast by our own instance of CUPS iptables -A INPUT -i $IFACE -s $IPADDR -d $BROADCAST -p udp --dport ipp --sport ipp -j DROP # Special case: silently drop DHCP request and renewal packets iptables -A INPUT -i $IFACE -s "0.0.0.0" -d "255.255.255.255" -p udp --sport bootpc --dport bootps -j DROP iptables -A INPUT -i $IFACE -s $ROUTER -d "255.255.255.255" -p udp --sport bootps --dport bootpc -j DROP # Refuse spoofed packets pretending to be from your IP address. iptables -A INPUT -i $IFACE -s $IPADDR -j log-and-drop # Refuse packets claiming to be from a Class A private network. iptables -A INPUT -i $IFACE -s $CLASS_A -j log-and-drop # Refuse packets claiming to be from a Class B private network. iptables -A INPUT -i $IFACE -s $CLASS_B -j log-and-drop # Refuse packets claiming to be from a Class C private network. # Note that we want to allow some, because we're ON a Class C network (hence the sub-table). # Packets from our LAN are passed back to the main INPUT table. # Other packets from Class C networks get dropped. iptables -N class-c-filter iptables -A INPUT -i $IFACE -s $CLASS_C -j class-c-filter iptables -A class-c-filter -i $IFACE -s $LAN -j RETURN iptables -A class-c-filter -i $IFACE -j log-and-drop # Refuse Class D multicast addresses. Multicast is illegal as a source address. iptables -A INPUT -i $IFACE -s $CLASS_D_MULTICAST -j log-and-drop # Refuse Class E reserved IP addresses. iptables -A INPUT -i $IFACE -s $CLASS_E_RESERVED_NET -j log-and-drop # Refuse packets claiming to be to the loopback interface. # Refusing packets claiming to be to the loopback interface protects against # source quench, whereby a machine can be told to slow itself down by an icmp source # quench to the loopback. iptables -A INPUT -i $IFACE -d $LOOPBACK -j log-and-drop # Refuse broadcast address packets, except for broadcasts across our LAN # iptables -A INPUT -i $IFACE -d $BROADCAST -j log-and-drop iptables -N broadcast-filter iptables -A INPUT -i $IFACE -d $BROADCAST -j broadcast-filter iptables -A broadcast-filter -i $IFACE -s $LAN -j RETURN iptables -A broadcast-filter -i $IFACE -j log-and-drop ## ============================================================ ## SERVICES # Note that this is mainly for the services provided by this server: services # used are generally allowed through the OUTPUT chain and the return packets # are either ESTABLISHED or RELATED. ## DNS (port 53) # Allow UDP and TCP packets in for DNS server. iptables -A INPUT -i $IFACE -p udp -s $LAN --dport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport domain -j ACCEPT # Allow UDP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $NAMESERVER_3 --sport domain -j ACCEPT # Allow TCP packets in if they are a response from a nameserver. iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_1 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_2 --sport domain -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $NAMESERVER_3 --sport domain -j ACCEPT ## Allow SSH (port 22) iptables -A INPUT -i $IFACE -p tcp --dport ssh -j ACCEPT ## Allow WWW (ports 80 and 443) iptables -A INPUT -i $IFACE -p tcp --dport http -j ACCEPT iptables -A INPUT -i $IFACE -p tcp --dport https -j ACCEPT ## Allow TELNET (port 23) # iptables -A INPUT -i $IFACE -p tcp --dport telnet -j ACCEPT ## Allow FTP (port 21 for control, port 20 for data) # Allow ftp outbound. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp -j ACCEPT # Now for the connection tracking part of ftp. This is discussed more completely in my section # on connection tracking to be found here. # 1) Active ftp. # This involves a connection INbound from port 20 on the remote machine, to a local port # passed over the ftp channel via a PORT command. The ip_conntrack_ftp module recognizes # the connection as RELATED to the original outgoing connection to port 21 so we don't # need NEW as a state match. ## iptables -A INPUT -i $IFACE -p tcp --dport ftp-data -j ACCEPT # 2) Passive ftp. # This involves a connection outbound from a port >1023 on the local machine, to a port >1023 # on the remote machine previously passed over the ftp channel via a PORT command. The # ip_conntrack_ftp module recognizes the connection as RELATED to the original outgoing # connection to port 21 so we don't need NEW as a state match. # iptables -A INPUT -i $IFACE -p tcp --sport $UP_PORTS --dport $UP_PORTS -m state --state ESTABLISHED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ## MAIL # SMTP (port 25) iptables -A INPUT -i $IFACE -p tcp --dport smtp -j ACCEPT # IMAP (ports 143, 220, 993). Only accept connections from machines on this LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imap3 -j ACCEPT iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport imaps -j ACCEPT # POP3 (ports 110, 995) # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3 -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport pop3s -j ACCEPT ## SAMBA (UDP ports 137, 138; TCP ports 139, 445) # Only accept packets from the LAN # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-ns -j ACCEPT # iptables -A INPUT -i $IFACE -p udp -s $LAN --dport netbios-dgm -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport netbios-ssn -j ACCEPT # iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport microsoft-ds -j ACCEPT ## CUPS printing (port 631) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport ipp -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport ipp -j ACCEPT ## MySQL (port 3306) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport mysql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport mysql -j ACCEPT ## PostgreSQL (port 5432) # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport postgresql -j ACCEPT iptables -A INPUT -i $IFACE -p udp -s $LAN --dport postgresql -j ACCEPT ## MongoDB # Only accept packets from the LAN iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27017 -j ACCEPT # MongoDB iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27018 -j ACCEPT # Mongo shardsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 27019 -j ACCEPT # Mongo configsvr iptables -A INPUT -i $IFACE -p tcp -s $LAN --dport 28017 -j ACCEPT # Mongo web interface ## Git (port 9418) iptables -A INPUT -i $IFACE -p tcp --dport git -j ACCEPT ## AUTH server (port 113) # Reject ident probes with a tcp reset. # I need to do this for a broken mailhost that won't accept my mail if I just drop its ident probe. # iptables -A INPUT -i $IFACE -p tcp --dport auth -j REJECT --reject-with tcp-reset # ICMP # We accept icmp in if it is "related" to other connections (e.g a time exceeded (11) # from a traceroute) or it is part of an "established" connection (e.g. an echo reply (0) # from an echo-request (8)). # iptables -A INPUT -i $IFACE -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT # (not needed due to the default acceptance of ESTABLISHED packets above ### Log and drop # Log and drop all remaining input packets iptables -A INPUT -j log-and-drop # THE END === Iptables script === #!/bin/sh # Start/stop iptables firewall # ### BEGIN INIT INFO # Provides: firewall # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 2 3 4 5 # Default-Stop: S 0 1 6 # Short-Description: Firewall # Description: Sets up a firewall with iptables. # Firewall rules are at /etc/iptables.rules ### END INIT INFO test -f /sbin/iptables || exit 0 . /lib/lsb/init-functions IPTABLES_CONFIG=/etc/iptables.rules # check we have the right kernel version KERNELMAJ=`uname -r | sed -e 's,\..*,,'` KERNELMIN=`uname -r | sed -e 's,[^\.]*\.,,' -e 's,\..*,,'` if [ "$KERNELMAJ" -lt 2 ] ; then exit 0 fi if [ "$KERNELMAJ" -eq 2 -a "$KERNELMIN" -lt 3 ] ; then exit 0 fi if /sbin/lsmod 2>/dev/null |grep -q ipchains ; then # Don't do both exit 0 fi start() { # don't do squat if we don't have the script if [ -f $IPTABLES_CONFIG ]; then log_begin_msg "Applying iptables firewall rules:" $IPTABLES_CONFIG log_end_msg $? fi } stop() { log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? log_begin_msg "Resetting built-in chains to the default ACCEPT policy:" iptables -P INPUT ACCEPT && \ iptables -P FORWARD ACCEPT && \ iptables -P OUTPUT ACCEPT && \ log_end_msg $? } panic() { log_begin_msg "Changing target policies to DROP: " iptables -P INPUT DROP && \ iptables -P FORWARD DROP && \ iptables -P OUTPUT DROP && \ log_end_msg $? log_begin_msg "Flushing all iptable chains" iptables -F log_end_msg $? log_begin_msg "Removing user defined iptable chains" iptables -X log_end_msg $? } case "$1" in start) start ;; stop) stop ;; restart) # "restart" is really just "start" as this isn't a daemon, # and "start" clears any pre-defined rules anyway. # This is really only here to make those who expect it happy panic start ;; status) iptables --list ;; panic) panic ;; *) echo "Usage: $0 {start|stop|restart|status|panic}" exit 1 esac exit 0 [[Category:Server setup]] 9fb34493afd9d8cb65706076af7f42e73126b398 2804 2797 2016-12-31T13:23:32Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} I used to do [[#Server (Old)|firewall setup manually]] for the server, but <code>ufw</code> has now improved to the point where I use that. See historic versions of this page from before December 2016 for details. ==Set up ufw== Enable ufw and add rules for the services the server machine will provide: root@server:~# ufw enable root@server:~# ufw allow ssh root@server:~# ufw allow http root@server:~# ufw allow https root@server:~# ufw allow domain root@server:~# ufw allow ipp root@server:~# ufw allow imap root@server:~# ufw allow smtp root@server:~# ufw allow submission root@server:~# ufw allow from 192.168.1.0/24 to any port mysql root@server:~# ufw allow from 192.168.1.0/24 to any port postgresql root@server:~# ufw allow git root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 27017:27019 # Mongo root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 28017 # Mongo web interface Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 (The last two rules are for [https://community.kde.org/KDEConnect KDE Connect] and [https://github.com/MatejVancik/amaroKontrol amaroKontrol] respectively). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 1714:1764/tcp ALLOW IN 192.168.1.0/24 8484/tcp ALLOW IN 192.168.1.0/24 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) [[Category:Server setup]] 7c15d133f1c1c3e30bf8caed05dc4d8b4feb6966 0 1544 2798 2623 2016-12-30T14:33:06Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# aptitude install bind9 :This will install the configuration files in <code>/etc/bind</code> * Alter the <code>/etc/bind/named.conf.options</code> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; dnssec-validation auto; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; :if you have problems with your forwarders not implementing DNSSEC (<code>error (insecurity proof failed) resolving './NS/IN'</code> appearing in <code>/var/log/syslog</code> and intermittent DNS service), you may want to replace the dnssec lines with: // dnssec-validation auto; dnssec-enable no; dnssec-validation no; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <code>/etc/bind/named.conf.local</code> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <code>/var/cache/bind</code> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <code>/etc/bind/</code>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <code>/etc/bind/db.domain.tld</code> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <code>/etc/bind/db.1.168.192</code> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update <code>/etc/resolv.conf</code> to ask the localhost for nameservers search domain.tld nameserver 127.0.0.1 nameserver 192.168.1.251 nameserver 192.168.1.252 :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Restart bind: root@server:~# systemctl restart bind9.service :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. For DD-WRT, the settings to use are: * '''Router IP''' ** ''Local DNS:'' 192.168.0.252 * '''Network Address Server Settings (DHCP)''' ** ''Static DNS 1:'' 192.168.0.252 ** ''Static DNS 2:'' 192.168.0.251 ** ''Static DNS 3:'' 0.0.0.0 ** ''WINS:'' 0.0.0.0 ** ''Use DNSMasq for DHCP:'' OFF ** ''Use DNSMasq for DNS:'' OFF ** ''DHCP-Authoritative:'' OFF == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <code>/usr/share/doc/bind9/README.Debian.gz</code> file on your machine's local disk has some good pointers. [[Category:Server setup]] 9a901b7ac4030378d093be76a01dbe715b386584 0 1570 2799 1961 2016-12-30T14:33:27Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[DNS server setup|DNS]] |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |} NTP (network time protocol) is a way of sending time across the internet. If a machine has an NTP client, it will keep its clock synchronised with various atomic clocks. The default <code>ntupdate</code> system synchronises the clock on every boot, but for long-running machines the clock can drift quite a bit between boots. A better option is to install the <code>ntp</code> daemon, which reset the clock every few minutes to prevent it getting too far adrift. * Install the daemon root@server:~# aptitude install ntp * Update <code>/etc/ntp.conf</code> to point to nearby servers server ntp.ubuntu.com server uk.pool.ntp.org :(find a nearby server from the [http://www.pool.ntp.org list of NTP server pools]). * Restart the <code>ntp</code> daemon so it knows about the new server root@server:~# systemctl restart ntp * If the clock is a few minutes off, it may confuse other server processes when <code>ntp</code> resets it the first time, so you may need to restart those servers. It may be easier to just reboot the machine. == See also == * This is pretty much taken from [https://help.ubuntu.com/8.04/serverguide/C/NTP.html the notes in the Ubuntu server guide]. * The [http://support.ntp.org/bin/view/Support/WebHome NTP documentation] has more information. [[Category:Server setup]] c9c938eb3ed00387b59e40f8a81c1431b9abcb47 0 1531 2807 2806 2016-12-31T15:18:42Z Neil 2 /* Amavis + ClamAV + SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering # # Default SPAM checking mode # Uncomment the two lines below to enable it back # @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); :(don't enable virus checking yet, as ClamAV 0.90, as included in Ubuntu 7.04, doesn't seem to work very well) * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# service amavis restart === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] b0679a1deaceb40f20733d6a1e30e0f4479f6f24 2808 2807 2016-12-31T16:22:15Z Neil 2 /* Amavis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# service spamassassin start === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 4a591c27abee696b76864abf86eb44b28c9ded2e 2809 2808 2016-12-31T16:22:33Z Neil 2 /* SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 2670d05b7e0baf1c1e2afde8901187c7121e56f3 2810 2809 2016-12-31T16:28:39Z Neil 2 /* Postfix */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/defaults/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# service opendkim restart root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] d49e8552f316e40ebff5ce682a666cba6845a64e 2811 2810 2016-12-31T17:14:28Z Neil 2 /* DKIM */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 SOCKET="inet:8891@localhost" # listen on loopback on port 8891 * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] f3fa408cf83ea455f7cd954cd3d06342e0344fbe 2812 2811 2016-12-31T17:20:18Z Neil 2 /* DKIM */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] ca9080ff04c61eff5af8230677870cceb7989949 2813 2812 2016-12-31T17:20:33Z Neil 2 /* DKIM */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/<code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# service opendmarc restart root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 046674a5675d482ea1a65112435c73278e5df0ea 2814 2813 2016-12-31T17:47:07Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code> to uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 9ccb8ebea4f0b915f1cafbe9bee63682345e4786 2815 2814 2016-12-31T18:13:01Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis smtp-amavis unix - - - - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 79518482448fee8001cb12f8837617c956125c23 2816 2815 2016-12-31T18:19:46Z Neil 2 /* Postfix */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 7fc9048836c04c338b82f3780b975ece93903f90 2817 2816 2016-12-31T18:20:38Z Neil 2 /* Basic installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav The remainder of ClamAV's default settings are adequate === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 12a6d3035d946f54bdfa31e6584730d83c5425a2 2820 2817 2017-01-02T15:36:47Z Neil 2 /* ClamAV */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] b10d3ac8360b497a3596e597f2fa7a963d3234d6 2838 2820 2017-01-04T15:58:20Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Sending Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 5e20d84574ad4f7ceca271060a7f9edaf15b72c3 2852 2838 2017-01-08T17:27:55Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL.) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Sending Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] f40ddd93163ab7b6ceb1411fdf963dabb43fb124 2853 2852 2017-01-08T17:29:55Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL.) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Sending Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] adc595606c67c890e7e383fed5152cf243a45fc8 0 1545 2818 2777 2016-12-31T20:57:39Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Desktop mail forwarding]] |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL]] |} If MySQL isn't already installed, install it: root@server:~# apt-get install mysql-server mysql-client == Passwords == The first thing is to add passwords to the MySQL accounts. <tt>root</tt> accounts for MySQL are created with no passwords (but note that you're asked for them even during a graphical package install). The [http://dev.mysql.com/doc/refman/5.0/en/index.html MySQL documentation] has [http://dev.mysql.com/doc/refman/5.0/en/default-privileges.html full instructions] for how to do this, but in summary the steps are: * Add passwords to the <code>root</code> accounts root@server:~# mysql -u root -p mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd'); mysql> SET PASSWORD FOR 'root'@'127.0.0.1' = PASSWORD('newpwd'); == Remote access == If you want to access MySQL databases from a remote machine, you'll need to have MySQL accept connections across the network. Edit <code>/etc/mysql/mysql.conf.d/mysql.cnf</code> and change the <code>bind-address</code> entry to be the IP of the machine: ==== Note to self: use machine name, not IP number ==== bind-address = server.domain.tld The restart MySQL: root@server:~# systemctl restart mysql.service Note that the [[Firewall setup|firewall]] is configured to only allow connections to MySQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [http://dev.mysql.com/doc/refman/5.7/en/index.html MySQL documentation] [[Category:Server setup]] 3046b5ebbac2d36d37514261c599904bb9494f28 0 1631 2819 2605 2016-12-31T21:32:13Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client On the desktop, install <code>pgadmin3</code> as well: root@desktop:~# aptitude install postgresql postgresql-client pgadmin3 == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf</code> and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/9.1/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] 00202e608e3532baa7e8c3d8fce7ea7ca317eff3 0 1534 2821 2719 2017-01-02T17:21:53Z Neil 2 /* Install the packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * For each wiki, create a directory where the wiki's files will sit: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki :Note that the mediawiki's pages will have the <tt>mediawiki</tt> in their path, but we'll get rid of that in a moment. Don't be tempted to install Mediawiki in the document root. * Create the symlinks for all the shared parts of Mediawiki: root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/api.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /var/lib/mediawiki/extensions/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/img_auth.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/includes/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/index.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/install-utils.inc root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/languages/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/maintenance/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/opensearch_desc.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/profileinfo.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/redirect.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/skins/ root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/StartProfiler.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/thumb.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/trackback.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/wiki.phtml root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/load.php root@server:/var/www/site.domain.tld/mediawiki# ln -s /usr/share/mediawiki/resources/ * Create copies of the files that need to be unique to this wiki root@server:/var/www/site.domain.tld/mediawiki# cp -r /usr/share/mediawiki/images . root@server:/var/www/site.domain.tld/mediawiki# cp -r /var/lib/mediawiki/config . root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data images root@server:/var/www/site.domain.tld/mediawiki# chown -R www-data:www-data config * Now modify the Apache configuration file for this site, <tt>/etc/apache2/sites-available/site.domain.tld</tt>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# /etc/init.d/apache2 reload * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. Make sure that you give each wiki a distinct database name. * Copy the <code>config/LocalSettings.php</code> file into the mediawiki root: root@server:/var/www/site.domain.tld/mediawiki# cp config/LocalSettings.php .ka * Ensure that the wiki program knows where the files are: modify <tt>/var/www/site.domain.tld/mediawiki/LocalSettings.php</tt> to include the line $IP = "/var/www/site.domain.tld/mediawiki"; :just after the default definition of <code>$IP</code> at the top of the file. * With the wiki set up, remove the <code>config</code> directory root@server:/var/www/site.domain.tld/mediawiki# rm -r /var/lib/mediawiki1.7/config That should be everything needed to get the wiki working. * Modify the <code>LocalSettings.php</code> file. The changes are marked below with <code>###</code> # $IP = "/var/lib/mediawiki1.7"; $IP = "/var/www/site.domain.tld/mediawiki"; ### # If PHP's memory limit is very low, some operations may fail. #ini_set( 'memory_limit', '20M' ); ini_set( 'memory_limit', '128M' ); ### $wgScriptPath = "/mediawiki"; ### $wgScript = "$wgScriptPath/index.php"; $wgRedirectScript = "$wgScriptPath/redirect.php"; ## If using PHP as a CGI module, the ?title= style usually must be used. # $wgArticlePath = "$wgScript/$1"; # Use this for the re-written, pretty URIs # $wgArticlePath = "$wgScript?title=$1"; $wgArticlePath = "/$1"; ### $wgUsePathInfo = true; ### $wgStylePath = "$wgScriptPath/skins"; $wgStyleDirectory = "$IP/skins"; $wgLogo = "$wgStylePath/common/images/wiki.png"; == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "/mediawiki/images/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 47ed6201b860e8a419c980eb8216b03ec44fc185 2822 2821 2017-01-02T17:29:36Z Neil 2 /* Creating the farm */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <tt><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></tt> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. Add the lines just below the <code>$wgEnableUploads</code> assignment. Note that the definition of the logo file is moved to after the definition of the upload directory pathname. $wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; $wgLogo = "/mediawiki/images/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 20e520ccd843d6ec3cccd4f436d1fefd73d3c690 2823 2822 2017-01-02T17:31:54Z Neil 2 /* Adding a logo */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Given the <code>mediawiki-math</code> pagkage is installed, it can be enabled with a simple edit of <code>LocalSettings.php</code>: $wgUseTeX = true; === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 4fd454e38a742207d1aabc6ad2f2660b7ae469a0 2824 2823 2017-01-02T17:34:52Z Neil 2 /* Enabling equation display */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <tt>LocalSettings.php</tt> by adding the <tt>$wgArticlePath</tt> and <tt>$wgUsePathInfo</tt> settings below: $wgScriptPath = "/mediawiki"; $wgScriptExtension = ".php"; # Site-specific settings $wgArticlePath = "/$1"; $wgUsePathInfo = false; (apparently, this is because of [http://www.mediawiki.org/wiki/Manual:Short_URL/Page_title_--_Working_method_with_mediawiki-1.11 a bug in Mediawiki 1.11]) You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] d2ee8676c3ab813060fc43f2e12f7fd2bb0aa35a 2825 2824 2017-01-02T17:38:34Z Neil 2 /* Rewriting paths */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MediaWiki:Monobook.css", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 3550ec4539442f2ef0225c1c60351726c776cd2c 2826 2825 2017-01-02T17:39:19Z Neil 2 /* Restricting viewing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/msword doc xls ppt # Added xls, ppt application/x-gzip gz tgz # Added tgz application/zip zip jar xpi sxc stc sxd std sxi sti sxm stm sxw stw odt ott oth odm odg otg odp otp ods ots odc odf odb odi oxt # Added od* and ot* (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] ef90e48145691bf791e9f4794505a10fbc4f9a43 2827 2826 2017-01-02T17:46:49Z Neil 2 /* Extending uploadable files */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 4e3b9ef51f0452549dabb4ac23c783e0abd7dcf4 2835 2827 2017-01-04T14:12:39Z Neil 2 /* Install the packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] d96208031a00ce4a889592b45ecfda61125bbba7 2848 2835 2017-01-07T14:32:39Z Neil 2 /* Creating the farm */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir /tmp/site.domain.tld/ root@server:~# chown www-data:www-data /tmp/site.domain.tld/ * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/tmp/domain.tld"; That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] a735b5b0cb3a9c0708dc934f7c3f95329aa702a0 2849 2848 2017-01-07T14:51:25Z Neil 2 /* Tweaking Mediawiki */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir /tmp/site.domain.tld/ root@server:~# chown www-data:www-data /tmp/site.domain.tld/ * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/tmp/domain.tld"; That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server https://www.domain.tld \ --urlpath http://www.domain.tld/sitemap \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 56528ca62d02ea5a4027109e9654e1081ac01a35 2850 2849 2017-01-07T14:52:17Z Neil 2 /* Generate sitemaps */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir /tmp/site.domain.tld/ root@server:~# chown www-data:www-data /tmp/site.domain.tld/ * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/tmp/domain.tld"; That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>http://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 84976d290290bc28d7856e60f8779f800fb98276 2851 2850 2017-01-07T15:10:31Z Neil 2 /* Generate sitemaps */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir /tmp/site.domain.tld/ root@server:~# chown www-data:www-data /tmp/site.domain.tld/ * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/tmp/domain.tld"; That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 1f8bb194233ed263194f7d7e0de0b12fda57643d 0 1533 2828 2774 2017-01-02T19:03:17Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# /etc/init.d/apache2 reload ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# service apache2 reload == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# /etc/init.d/apache2 reload == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/cert.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# /etc/init.d/apache2 restart Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ===Note to self=== On Apache version 2.4.8 and above, replace: SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem with: SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off ==Add modules needed== root@server:~# a2enmod headers root@server:~# a2enmod cgi root@server:~# systemctl restart apache2.service == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] ed075ff90342e98bca2d2355cefe16f268fc345d 0 1594 2829 2728 2017-01-02T23:38:16Z Neil 2 /* Installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ * Create a new user for <code>awstats</code>, make it part of the <code>www-data</code> and <code>adm</code> groups. Give it permissions for the awstats data files. root@desktop:~# useradd -g www-data -G adm awstats root@desktop:~# chown -R awstats:www-data /var/lib/awstats * Make the cron jobs run as this user. Adjust <code>/etc/cron.d/awstats</code> to change the <code>www-data</code> user to <code>awstats</code> MAILTO=root */10 * * * * awstats [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * awstats [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh == AWStats configuration == ==SiteDomain== AWStats doesn't like not knowing a default domain, even though the domain is specified in each site-specific configuration below. Modify <code>/etc/awstats/awstats.conf</code> to include a default domain, such as the one that serves the AWStats pages. SiteDomain="site.domain.tld" === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# service apache2 reload == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 82ab6a4a1d02602391855da5d4b155a136600387 2831 2829 2017-01-03T14:52:52Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ * Create a new user for <code>awstats</code>, make it part of the <code>www-data</code> and <code>adm</code> groups. Give it permissions for the awstats data files. root@desktop:~# useradd -g www-data -G adm awstats root@desktop:~# chown -R awstats:www-data /var/lib/awstats * Make the cron jobs run as this user. Adjust <code>/etc/cron.d/awstats</code> to change the <code>www-data</code> user to <code>awstats</code> MAILTO=root */10 * * * * awstats [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * awstats [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh == AWStats configuration == ==SiteDomain== AWStats doesn't like not knowing a default domain, even though the domain is specified in each site-specific configuration below. Modify <code>/etc/awstats/awstats.conf</code> to include a default domain, such as the one that serves the AWStats pages. SiteDomain="site.domain.tld" === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# systemctl reload apache2.service == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == Cataloguing old stats and logs == If you want to keep historic stats from an old machine when moving to a new one, you should be able to move the files in <code>DirData</code> in the config files, typically <code>/var/lib/awstats</code>. However, that didn't work for me, so I recreated the stats from the old log files. * Disable automatic updates while you're doing this. Edit <code>/etc/cron.d/awstats</code> and comment out the two commands there. * Merge all the logs you want into one file: root@desktop:~# perl /usr/share/awstats/tools/logresolvemerge.pl /var/log/apache2/site.access.log* > site.all.log * Process them (this will take a long time): root@desktop:~# /var/www/cgi-bin.scripts.domain.tld/awstats.pl -update -config=/etc/awstats/awstats.domain.tld.conf LogFile=site.all.log == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] d43dba1ba6df44fd5f52080c3782be2143d12d5d 2834 2831 2017-01-04T13:23:36Z Neil 2 /* Cataloguing old stats and logs */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ * Create a new user for <code>awstats</code>, make it part of the <code>www-data</code> and <code>adm</code> groups. Give it permissions for the awstats data files. root@desktop:~# useradd -g www-data -G adm awstats root@desktop:~# chown -R awstats:www-data /var/lib/awstats * Make the cron jobs run as this user. Adjust <code>/etc/cron.d/awstats</code> to change the <code>www-data</code> user to <code>awstats</code> MAILTO=root */10 * * * * awstats [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * awstats [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh == AWStats configuration == ==SiteDomain== AWStats doesn't like not knowing a default domain, even though the domain is specified in each site-specific configuration below. Modify <code>/etc/awstats/awstats.conf</code> to include a default domain, such as the one that serves the AWStats pages. SiteDomain="site.domain.tld" === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# systemctl reload apache2.service == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == Cataloguing old stats and logs == If you want to keep historic stats from an old machine when moving to a new one, you should be able to move the files in <code>DirData</code> in the config files, typically <code>/var/lib/awstats</code>. However, that didn't work for me, so I recreated the stats from the old log files. * Disable automatic updates while you're doing this. Edit <code>/etc/cron.d/awstats</code> and comment out the two commands there. * Merge all the logs you want into one file: root@desktop:~# perl /usr/share/awstats/tools/logresolvemerge.pl /var/log/apache2/site.access.log* > site.all.log * Process them (this will take a long time): root@desktop:~# /var/www/cgi-bin.scripts.domain.tld/awstats.pl -update -config=domain.tld LogFile=site.all.log == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 7699c96a5e1315042ddcb53387eb682681a2aa08 0 1544 2830 2798 2017-01-03T10:37:19Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[NTP client setup|NTP]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# aptitude install bind9 :This will install the configuration files in <code>/etc/bind</code> * Alter the <code>/etc/bind/named.conf.options</code> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; dnssec-validation auto; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; :if you have problems with your forwarders not implementing DNSSEC (<code>error (insecurity proof failed) resolving './NS/IN'</code> appearing in <code>/var/log/syslog</code> and intermittent DNS service), you may want to replace the dnssec lines with: // dnssec-validation auto; dnssec-enable no; dnssec-validation no; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <code>/etc/bind/named.conf.local</code> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <code>/var/cache/bind</code> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <code>/etc/bind/</code>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <code>/etc/bind/db.domain.tld</code> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <code>/etc/bind/db.1.168.192</code> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update <code>/etc/network/interfaces</code> to tell <code>dnsmasq</code> about the nameservers # The primary network interface auto enp0s7 iface enp0s7 inet static address 192.168.1.251 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * Restart bind: root@server:~# systemctl restart bind9.service :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. For DD-WRT, the settings to use are: * '''Router IP''' ** ''Local DNS:'' 192.168.0.252 * '''Network Address Server Settings (DHCP)''' ** ''Static DNS 1:'' 192.168.0.252 ** ''Static DNS 2:'' 192.168.0.251 ** ''Static DNS 3:'' 0.0.0.0 ** ''WINS:'' 0.0.0.0 ** ''Use DNSMasq for DHCP:'' OFF ** ''Use DNSMasq for DNS:'' OFF ** ''DHCP-Authoritative:'' OFF == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <code>/usr/share/doc/bind9/README.Debian.gz</code> file on your machine's local disk has some good pointers. [[Category:Server setup]] 9ee3c8e23fb4aa7d521bd10394943b07da2b403d 0 1542 2832 2720 2017-01-04T12:53:47Z Neil 2 /* Update Avahi demon */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. ==Allow network printing== Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. ===Update Avahi demon=== The Avahi demon may not allow printing to networked printers, as the client machines may not understand the ''hostname''.local naming convention that CUPS expects. On Ubuntu 16.10 machines, modify <code>/etc/nsswitch.conf</code> so the <code>hosts:</code> line reads: hosts: files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns myhostname On all machines, add <code>myhostname</code> to the end of that line. Restart Avahi and CUPS: root@server:~# service avahi-daemon restart root@server:~# service cups restart == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it, as described on the [[Scanner setup]] page. ==Bugfix== CUPS on Ubuntu 12.04 has a [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/872483 bug that affects the USB backend], preventing the printer printing more than one job with it needing resetting. Once the printer is set up, issue the command: root@server:~# lpadmin -p <printer> -o usb-no-reattach-default=true == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] 3eb3959add8d130e5302203f21a94228f282874a 2833 2832 2017-01-04T13:06:10Z Neil 2 /* Allow network printing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. ==Allow network printing== Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. '''Not to self''' The HP LaserJet 1320 printer I have doesn't like the "recommended" Gutenprint driver. Instead, configure it to use the <code>HP LaserJet 1320 series Postscript (recommended):q I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. ===Update Avahi demon=== The Avahi demon may not allow printing to networked printers, as the client machines may not understand the ''hostname''.local naming convention that CUPS expects. On Ubuntu 16.10 machines, modify <code>/etc/nsswitch.conf</code> so the <code>hosts:</code> line reads: hosts: files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns myhostname On all machines, add <code>myhostname</code> to the end of that line. Restart Avahi and CUPS: root@server:~# service avahi-daemon restart root@server:~# service cups restart == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it, as described on the [[Scanner setup]] page. ==Bugfix== CUPS on Ubuntu 12.04 has a [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/872483 bug that affects the USB backend], preventing the printer printing more than one job with it needing resetting. Once the printer is set up, issue the command: root@server:~# lpadmin -p <printer> -o usb-no-reattach-default=true == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] cd690f478a2a9fd0c4af0ff26e98ef01f97b67ff 2837 2833 2017-01-04T15:39:58Z Neil 2 /* Allow network printing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Webmail setup|Webmail]] |align="center" width="50%"| [[Scanner setup|Scanner]] |} CUPS is easy to set up. ==Allow network printing== Out of the box, CUPS only allows connections from <code>localhost</code>. That's OK for the desktop machine, but is a bit of a problem for setting up the server. Modify <code>/etc/cups/cupsd.conf</code> at the top to look like this: ## Only listen for connections from the local machine. # Listen localhost:631 # Listen for connections from anywhere Port 631 Listen /var/run/cups/cups.sock ServerAlias desktop.domain.tld ServerAlias printer.domain.tld (the <code>ServerAlias</code> line is needed to allow the server to respond to remote requests. Repeat this line for each alias the CUPS server responds to. See [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/516018 Ubuntu bug 516018] for details.) Add the <code>Allow @LOCAL</code> lines in the two <code>Location</code> stanzas in the same file: <Location /> # Allow shared printing and remote administration... Order allow,deny Allow @LOCAL </Location> <Location /admin> # Allow remote administration... Order allow,deny Allow @LOCAL </Location> then restart CUPS: root@server:~# service cups restart You should then be able to point a webbrowser at <code>desktop.server.tld:631</code> or <code>server.domain.tld:631</code> and configure printers via the web interface. As I have printers attached to both machines, and I want various Windows boxes to use the printers, I need to publish the printer details. On the <code>Administration</code> page, select these options: * Show printers shared by other systems * Share published printers connected to this system * Allow printing from the Internet * Allow remote administration * Allow users to cancel any job (not just their own) Do that on both machines, and you'll be able to access either printer from either machine. '''Note to self''' The HP LaserJet 1320 printer I have doesn't like the "recommended" Gutenprint driver. Instead, configure it to use the <code>HP LaserJet 1320 series Postscript (recommended)</code>. I also installed the <code>hplip</code> package and ran <code>hp-setup -i</code> . I had trouble using Chrome for the administration (I think it couldn't handle the certificates), so I used ReKonq instead. ===Update Avahi demon=== The Avahi demon may not allow printing to networked printers, as the client machines may not understand the ''hostname''.local naming convention that CUPS expects. On Ubuntu 16.10 machines, modify <code>/etc/nsswitch.conf</code> so the <code>hosts:</code> line reads: hosts: files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns myhostname On all machines, add <code>myhostname</code> to the end of that line. Restart Avahi and CUPS: root@server:~# service avahi-daemon restart root@server:~# service cups restart == Updating HPLIP == I've got a multi-function printer (HP 1312). Scanning for this printer is not supported in the HPLIP that comes with Ubuntu. So I need to update it, as described on the [[Scanner setup]] page. ==Bugfix== CUPS on Ubuntu 12.04 has a [https://bugs.launchpad.net/ubuntu/+source/cups/+bug/872483 bug that affects the USB backend], preventing the printer printing more than one job with it needing resetting. Once the printer is set up, issue the command: root@server:~# lpadmin -p <printer> -o usb-no-reattach-default=true == Windows use of printers == To allow Windows boxes to use the printers, you'll first need to create a 'raw' queue for each printer you want to make available. Then create a new printer in Windows. In the wizard, ask for a 'Printer connected to the internet', give the address as <code><nowiki>http://server.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>, and install the Windows driver for that printer on the Windows box. That should be it! For more detailed instructions on using CUPS with Windows, see this [http://www.owlfish.com/thoughts/winipp-cups-2003-07-20.html How to make Windows use CUPS IPP] page. * Start a web browser and point it at <code><nowiki>http://printers.domain.tld:631/printers</nowiki></code>. Select the printer you want, such as <code><nowiki>http://printers.domain.tld:631/printers/RawPrinterQueueName</nowiki></code>. Copy this address to the clipboard. * Go to Control Panel > Add a new printer. Add a network printer. Don't bother searching for the printer, simply paste in the address you copied above. * Install the printer drivers as normal. You should now have a working printer available from CUPS. [[Category:Server setup]] b3806f59a085f59af716b409cd7b6de48d4db560 0 1593 2836 2794 2017-01-04T15:15:31Z Neil 2 /* Install and setup git-daemon */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon-run * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf-available/gitweb.conf</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options +FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Enable the gitweb config and reload Apache: root@server:~# a2enconf gitweb root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Changing the default template== Modify the standard template if you want changes to affect all new repos created. Do this, for instance, if you want to use [https://keybase.io/neilnjae Keybase.io] to automatically sign all commits by adding the file <code>hooks/pre-commit</code>: #!/bin/sh # Run Keybase to sign the commit keybase dir sign -p git git add SIGNED.md ===Server side=== Modify the contents of <code>/usr/share/git-core/templates/</code> as you see fit, such as adding the commit hook above. This should affect all repos created with Gitolite. ===Client side=== Copy <code>/usr/share/git-core/templates/</code> to somewhere local, such as <code>~/.git_template</code>. Modify the contents as you see fit. Tell Git to use this template rather than the default one: user@desktop:~$ git config --global init.templatedir '~/.git_template' This should affect all new repos created on the client, outside of Gitolote. ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create its entry in <code>gitolite-admin/conf/gitolite.conf</code> and push the changes. This will create an empty repo on the server. Then clone it on the client machine, ignoring the message about cloning an empty repo. * To import a repo into Github, use the Github "import repository" functionality, using the url <code><nowiki>git://git.domain.tld/repo.git</nowiki></code> . Note that you may need to remove files larger than 100Mb, following [https://help.github.com/articles/remove-sensitive-data/ Github's instructions]. * To add Github as a remote for a repo, git remote add github git@github.com:neilnjae/foo.git git push github master ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 542ad23ebecd715daa9dc02163cf5f7f88a5dded 0 1647 2839 2017-01-04T16:56:38Z Neil 2 Created page with "* Create the <code>dmarc</code> database user. root@server:~# sudo -u postgres psql postgres=# create user dmarc with login encrypted password 'xxx'; * Create the confi..." wikitext text/x-wiki * Create the <code>dmarc</code> database user. root@server:~# sudo -u postgres psql postgres=# create user dmarc with login encrypted password 'xxx'; * Create the config file, <code>/etc/dmarc_to_database.ini</code>, and give it <code>600</code> permissions. [DEFAULT] [imap] server = imap.domain.tld username = dmarc@domain.tld port = 143 password = xxx [database] server = server.domain.tld database = dmarc username = dmarc password = xxx * Create the database to store the Dmarc reports. root@ogedei:/var/lib/mediawiki# sudo -u postgres createdb --owner=dmarc dmarc "Dmarc reports to send to originators" * Run the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=make_database.py make_database.py]</code> script to create the tables in the database. You can run this on any machine that can see the Postgres server; you may need another copy of the <code>dmarc.ini</code> file. * Copy the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=make_database.py make_database.py]</code> script to <code>/etc/cron.daily/</code> and make it executable. e61ac2fab6c9b65961bcd34444150047052c110a 2840 2839 2017-01-04T16:58:26Z Neil 2 wikitext text/x-wiki * Create the <code>dmarc</code> database user. root@server:~# sudo -u postgres psql postgres=# create user dmarc with login encrypted password 'xxx'; * Create the config file, <code>/etc/dmarc_to_database.ini</code>, and give it <code>600</code> permissions. [DEFAULT] [imap] server = imap.domain.tld username = dmarc@domain.tld port = 143 password = xxx [database] server = server.domain.tld database = dmarc username = dmarc password = xxx * Create the database to store the Dmarc reports. root@ogedei:/var/lib/mediawiki# sudo -u postgres createdb --owner=dmarc dmarc "Dmarc reports to send to originators" * Run the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=make_database.py make_database.py]</code> script to create the tables in the database. You can run this on any machine that can see the Postgres server; you may need another copy of the <code>dmarc.ini</code> file. * Copy the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=dmarc_to_database dmarc_to_database]</code> script to <code>/etc/cron.daily/</code> and make it executable. fd6b9eac0c850a54e76a74975f5f5a8010f2fc04 2841 2840 2017-01-04T17:00:39Z Neil 2 wikitext text/x-wiki * Create the <code>dmarc</code> database user. root@server:~# sudo -u postgres psql postgres=# create user dmarc with login encrypted password 'xxx'; * Create the config file, <code>/etc/dmarc_to_database.ini</code>, and give it <code>600</code> permissions. [DEFAULT] [imap] server = imap.domain.tld username = dmarc@domain.tld port = 143 password = xxx [database] server = server.domain.tld database = dmarc username = dmarc password = xxx Create a <code>dmarc</code> email virtual user, according to the [[Dovecot_server_setup#Create_virtual_users|Dovecot instructions]]. * Create the database to store the Dmarc reports. root@ogedei:/var/lib/mediawiki# sudo -u postgres createdb --owner=dmarc dmarc "Dmarc reports to send to originators" * Run the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=make_database.py make_database.py]</code> script to create the tables in the database. You can run this on any machine that can see the Postgres server; you may need another copy of the <code>dmarc.ini</code> file. * Copy the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=dmarc_to_database dmarc_to_database]</code> script to <code>/etc/cron.daily/</code> and make it executable. 67f7b2329b3945a139c695fd3bcfc2e22406d5e5 2844 2841 2017-01-05T13:09:39Z Neil 2 wikitext text/x-wiki * Install the required libraries root@server:~# aptitude install python3 python3-pip libpq-dev root@server:~# pip3 install psycopg2 * Create the <code>dmarc</code> database user. root@server:~# sudo -u postgres psql postgres=# create user dmarc with login encrypted password 'xxx'; * Create the config file, <code>/etc/dmarc_to_database.ini</code>, and give it <code>600</code> permissions. [DEFAULT] [imap] server = imap.domain.tld username = dmarc@domain.tld port = 143 password = xxx [database] server = server.domain.tld database = dmarc username = dmarc password = xxx Create a <code>dmarc</code> email virtual user, according to the [[Dovecot_server_setup#Create_virtual_users|Dovecot instructions]]. * Create the database to store the Dmarc reports. root@ogedei:/var/lib/mediawiki# sudo -u postgres createdb --owner=dmarc dmarc "Dmarc reports to send to originators" * Run the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=make_database.py make_database.py]</code> script to create the tables in the database. You can run this on any machine that can see the Postgres server; you may need another copy of the <code>dmarc.ini</code> file. * Copy the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=dmarc_to_database dmarc_to_database]</code> script to <code>/etc/cron.daily/</code> and make it executable. c7ffffc1d574acfba72db6aca1a2d7796155cde5 2846 2844 2017-01-06T15:24:31Z Neil 2 Neil moved page [[Sending Dmarc feedback]] to [[Recording Dmarc feedback]] without leaving a redirect wikitext text/x-wiki * Install the required libraries root@server:~# aptitude install python3 python3-pip libpq-dev root@server:~# pip3 install psycopg2 * Create the <code>dmarc</code> database user. root@server:~# sudo -u postgres psql postgres=# create user dmarc with login encrypted password 'xxx'; * Create the config file, <code>/etc/dmarc_to_database.ini</code>, and give it <code>600</code> permissions. [DEFAULT] [imap] server = imap.domain.tld username = dmarc@domain.tld port = 143 password = xxx [database] server = server.domain.tld database = dmarc username = dmarc password = xxx Create a <code>dmarc</code> email virtual user, according to the [[Dovecot_server_setup#Create_virtual_users|Dovecot instructions]]. * Create the database to store the Dmarc reports. root@ogedei:/var/lib/mediawiki# sudo -u postgres createdb --owner=dmarc dmarc "Dmarc reports to send to originators" * Run the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=make_database.py make_database.py]</code> script to create the tables in the database. You can run this on any machine that can see the Postgres server; you may need another copy of the <code>dmarc.ini</code> file. * Copy the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=dmarc_to_database dmarc_to_database]</code> script to <code>/etc/cron.daily/</code> and make it executable. c7ffffc1d574acfba72db6aca1a2d7796155cde5 2847 2846 2017-01-06T15:25:18Z Neil 2 wikitext text/x-wiki * Install the required libraries root@server:~# aptitude install python3 python3-pip libpq-dev root@server:~# pip3 install psycopg2 * Create the <code>dmarc</code> database user. root@server:~# sudo -u postgres psql postgres=# create user dmarc with login encrypted password 'xxx'; * Create the config file, <code>/etc/dmarc_to_database.ini</code>, and give it <code>600</code> permissions. [DEFAULT] [imap] server = imap.domain.tld username = dmarc@domain.tld port = 143 password = xxx [database] server = server.domain.tld database = dmarc username = dmarc password = xxx Create a <code>dmarc</code> email virtual user, according to the [[Dovecot_server_setup#Create_virtual_users|Dovecot instructions]]. * Create the database to store the Dmarc reports. root@ogedei:/var/lib/mediawiki# sudo -u postgres createdb --owner=dmarc dmarc "Dmarc reports received from other sites" * Run the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=make_database.py make_database.py]</code> script to create the tables in the database. You can run this on any machine that can see the Postgres server; you may need another copy of the <code>dmarc.ini</code> file. * Copy the <code>[https://git.njae.me.uk/?p=dmarc.git;a=blob;f=dmarc_to_database dmarc_to_database]</code> script to <code>/etc/cron.daily/</code> and make it executable. b72e5704ad4e44caf280814be8df00927dc8614f 0 1535 2842 2464 2017-01-05T11:30:19Z Neil 2 Updated for Roundcube wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I'm using [https://roundcube.net/ Roundcube] as a webmail interface. * Install the packages: root@server:~# aptitude install roundcube roundcube-pgsql :(Install other <code>aspell-</code> dictionaries as you wish at the same time.) * Follow the defaults in the questions asked during installation. However, these questions don't address all the configuration options. Therefore, after installation, reconfigure the packages: root@server:~# sudo dpkg-reconfigure roundcube-core * Roundcube wants to make itself available as <code>domain.tld/roundcube</code> on all sites. The file <code>/etc/apache2/conf-available/roundcube.conf</code> does this with the <code>Alias /roundcube /var/lib/roundcube</code> directive, which is initially commented out. Keep it commented out, and copy the contents of that file into the <code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>, uncommenting the <code>Alias</code> line. * Disable the Roundcube configuration, enable the webmail site, and reload Apache: root@server:~# a2disconf roundcube root@server:~# a2ensite webmail.domain.tld.conf root@server:~# systemctl reload apache2.service * Visit <code><nowiki>https://webmail.domain.tld</nowiki></code> and you should be able to log on. ===<code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>=== <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On # SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/fullchain.pem DocumentRoot /var/lib/roundcube ServerName webmail.domain.tld <Directory /> Options +FollowSymLinks -Indexes AllowOverride None </Directory> <Directory /var/lib/roundcube/> Options +FollowSymLinks # This is needed to parse /var/lib/roundcube/.htaccess. See its # content before setting AllowOverride to None. AllowOverride All <IfVersion >= 2.3> Require all granted </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all </IfVersion> </Directory> # Protecting basic directories: <Directory /var/lib/roundcube/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/lib/roundcube/temp> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> <Directory /var/lib/roundcube/logs> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/webmail.domain.tld.access.log combined ServerSignature Off </VirtualHost> == See also == * [https://github.com/roundcube/roundcubemail/wiki Roundcube documentation] * [https://help.ubuntu.com/community/Roundcube Installing Roundcube on Ubuntu] [[Category:Server setup]] 5c076e33af690bfa7abbbcbb278594ab756b58ca 0 1515 2843 2779 2017-01-05T11:30:50Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 12.04 LTS and 13.10 (adapted from 8.04 LTS by way of 10.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Roundcube)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Desktop mail forwarding]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] acaedf7d951a297e144c44a1f9c794417e4b15ed 0 1540 2845 2791 2017-01-05T13:55:28Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] d155e6856404ff0613ea711d0837db86ad072834 0 1645 2854 2785 2017-01-08T17:38:03Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.org/en/latest/using.html#installation install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== This is a bit of chicken-and-egg. First, you need to create the non-HTTPS virtual sites for all the domains you want to serve. You can then use the Apache plugin to the LetsEncrypt client to fetch and install the certificates. Then, you can go back and fix the various certificate settings in the virtual host conf files. If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# letsencrypt --apache -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# letsencrypt --apache -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# letsencrypt --apache -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--expand</code> option and list all the domains for the certificate: root@server:~# /opt/letsencrypt/letsencrypt-auto -d domain.tld -d www.domain.tld -d other.domain.tld ==Check the contents of a certificate== If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text root@server:~# service apache2 reload Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==Automatically update certificates== Create the file <code>/etc/cron.daily/letsencrypt-renew</code>: #!/bin/sh /usr/bin/letsencrypt renew # --agree-tos systemctl reload apache2.service systemctl reload dovecot.service systemctl reload postfix.service Make it executable: root@server:# chmod +x /etc/cron.daily/letsencrypt-renew ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 4453c2730aaf2a2f7be8b1dc25ac31d7f0441e01 2855 2854 2017-01-08T17:38:40Z Neil 2 /* Get certificates from Lets Encrypt */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.org/en/latest/using.html#installation install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== This is a bit of chicken-and-egg. First, you need to create the non-HTTPS virtual sites for all the domains you want to serve. You can then use the Apache plugin to the LetsEncrypt client to fetch and install the certificates. Then, you can go back and fix the various certificate settings in the virtual host conf files. If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# letsencrypt --apache -d domain.tld -d www.domain.tld root@server:# letsencrypt --apache -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# letsencrypt --apache -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--expand</code> option and list all the domains for the certificate: root@server:~# /opt/letsencrypt/letsencrypt-auto -d domain.tld -d www.domain.tld -d other.domain.tld ==Check the contents of a certificate== If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text root@server:~# service apache2 reload Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==Automatically update certificates== Create the file <code>/etc/cron.daily/letsencrypt-renew</code>: #!/bin/sh /usr/bin/letsencrypt renew # --agree-tos systemctl reload apache2.service systemctl reload dovecot.service systemctl reload postfix.service Make it executable: root@server:# chmod +x /etc/cron.daily/letsencrypt-renew ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 6cc9d8c93333987a8cedc95b4e2f574b34a2c067 2856 2855 2017-01-08T17:39:39Z Neil 2 /* Extend the domains of a certificate */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.org/en/latest/using.html#installation install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== This is a bit of chicken-and-egg. First, you need to create the non-HTTPS virtual sites for all the domains you want to serve. You can then use the Apache plugin to the LetsEncrypt client to fetch and install the certificates. Then, you can go back and fix the various certificate settings in the virtual host conf files. If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# letsencrypt --apache -d domain.tld -d www.domain.tld root@server:# letsencrypt --apache -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# letsencrypt --apache -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--expand</code> option and list all the domains for the certificate: root@server:~# letsencrypt --expand -d domain.tld -d www.domain.tld -d other.domain.tld ==Check the contents of a certificate== If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text root@server:~# service apache2 reload Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==Automatically update certificates== Create the file <code>/etc/cron.daily/letsencrypt-renew</code>: #!/bin/sh /usr/bin/letsencrypt renew # --agree-tos systemctl reload apache2.service systemctl reload dovecot.service systemctl reload postfix.service Make it executable: root@server:# chmod +x /etc/cron.daily/letsencrypt-renew ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] b8d6ddba94eaa63756b44c1f93560b9c90b458a6 0 1594 2857 2834 2017-01-16T11:24:05Z Neil 2 /* Cataloguing old stats and logs */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ * Create a new user for <code>awstats</code>, make it part of the <code>www-data</code> and <code>adm</code> groups. Give it permissions for the awstats data files. root@desktop:~# useradd -g www-data -G adm awstats root@desktop:~# chown -R awstats:www-data /var/lib/awstats * Make the cron jobs run as this user. Adjust <code>/etc/cron.d/awstats</code> to change the <code>www-data</code> user to <code>awstats</code> MAILTO=root */10 * * * * awstats [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * awstats [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh == AWStats configuration == ==SiteDomain== AWStats doesn't like not knowing a default domain, even though the domain is specified in each site-specific configuration below. Modify <code>/etc/awstats/awstats.conf</code> to include a default domain, such as the one that serves the AWStats pages. SiteDomain="site.domain.tld" === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# systemctl reload apache2.service == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == Cataloguing old stats and logs == If you want to keep historic stats from an old machine when moving to a new one, you should be able to move the files in <code>DirData</code> in the config files, typically <code>/var/lib/awstats</code>. However, that didn't work for me, so I recreated the stats from the old log files. * Disable automatic updates while you're doing this. Edit <code>/etc/cron.d/awstats</code> and comment out the two commands there. * Merge all the logs you want into one file: root@desktop:~# perl /usr/share/awstats/tools/logresolvemerge.pl /var/log/apache2/site.access.log* > site.all.log * Process them (this will take a long time): root@desktop:~# /var/www/cgi-bin.scripts.domain.tld/awstats.pl -update -config=domain.tld -LogFile=site.all.log == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 2b5c3d36a9d8f7538509e6e58233a3e464de89d7 2858 2857 2017-01-16T17:16:59Z Neil 2 /* Cataloguing old stats and logs */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ * Create a new user for <code>awstats</code>, make it part of the <code>www-data</code> and <code>adm</code> groups. Give it permissions for the awstats data files. root@desktop:~# useradd -g www-data -G adm awstats root@desktop:~# chown -R awstats:www-data /var/lib/awstats * Make the cron jobs run as this user. Adjust <code>/etc/cron.d/awstats</code> to change the <code>www-data</code> user to <code>awstats</code> MAILTO=root */10 * * * * awstats [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * awstats [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh == AWStats configuration == ==SiteDomain== AWStats doesn't like not knowing a default domain, even though the domain is specified in each site-specific configuration below. Modify <code>/etc/awstats/awstats.conf</code> to include a default domain, such as the one that serves the AWStats pages. SiteDomain="site.domain.tld" === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# systemctl reload apache2.service == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == Cataloguing old stats and logs == If you want to keep historic stats from an old machine when moving to a new one, you should be able to move the files in <code>DirData</code> in the config files, typically <code>/var/lib/awstats</code>. However, that didn't work for me, so I recreated the stats from the old log files. * Disable automatic updates while you're doing this. Edit <code>/etc/cron.d/awstats</code> and comment out the two commands there. * Merge all the logs you want into one file: root@desktop:~# perl /usr/share/awstats/tools/logresolvemerge.pl /var/log/apache2/site.access.log* > site.all.log * Mail logs need to be manually strung together in date order, converted, and bundled together. root@desktop:~# rm mail.converted.log root@desktop:~# for f in `ls -tr /var/log/mail.log*` ; do /usr/share/doc/awstats/examples/maillogconvert.pl standard < $f >> mail.converted.log; done * Process them (this will take a long time): root@desktop:~# /var/www/cgi-bin.scripts.domain.tld/awstats.pl -update -config=domain.tld -LogFile=site.all.log == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] f77edf3a3f69905d0e0878566f58cb916367a726 0 1535 2859 2842 2017-01-22T11:22:51Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I'm using [https://roundcube.net/ Roundcube] as a webmail interface. * Install the packages: root@server:~# aptitude install roundcube roundcube-pgsql :(Install other <code>aspell-</code> dictionaries as you wish at the same time.) * Follow the defaults in the questions asked during installation. However, these questions don't address all the configuration options. Therefore, after installation, reconfigure the packages: root@server:~# sudo dpkg-reconfigure roundcube-core * Roundcube wants to make itself available as <code>domain.tld/roundcube</code> on all sites. The file <code>/etc/apache2/conf-available/roundcube.conf</code> does this with the <code>Alias /roundcube /var/lib/roundcube</code> directive, which is initially commented out. Keep it commented out, and copy the contents of that file into the <code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>, uncommenting the <code>Alias</code> line. * Disable the Roundcube configuration, enable the webmail site, and reload Apache: root@server:~# a2disconf roundcube root@server:~# a2ensite webmail.domain.tld.conf root@server:~# systemctl reload apache2.service * Visit <code><nowiki>https://webmail.domain.tld</nowiki></code> and you should be able to log on. ==Installing plugins== Where possible, use the [https://plugins.roundcube.net/ Roundcube plugins installer], working from the <code>/var/lib/roundcube</code> directory. ===Mobile skin=== Roundcube doesn't work well on mobile devices. The [https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile Melanie2 skin] seems to be a good alternative. The [https://roundcubeinbox.wordpress.com/2016/04/26/roundcube-for-mobile-devices/ instructions at the Roundcube inbox] are slightly unclear with paths, so here's what I did. root@server:~# cd /var/lib/roundcube root@server:/var/lib/roundcube# git clone https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile.git skins/melanie2_larry_mobile root@server:~# php composer.phar require melanie2/mobile:dev-master :remembering to say "no" to activating the <code>jquery_mobile</code> plugin, but "yes" to activating the <code>mobile</code> plugin. ==<code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>== <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On # SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/fullchain.pem DocumentRoot /var/lib/roundcube ServerName webmail.domain.tld <Directory /> Options +FollowSymLinks -Indexes AllowOverride None </Directory> <Directory /var/lib/roundcube/> Options +FollowSymLinks # This is needed to parse /var/lib/roundcube/.htaccess. See its # content before setting AllowOverride to None. AllowOverride All <IfVersion >= 2.3> Require all granted </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all </IfVersion> </Directory> # Protecting basic directories: <Directory /var/lib/roundcube/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/lib/roundcube/temp> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> <Directory /var/lib/roundcube/logs> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/webmail.domain.tld.access.log combined ServerSignature Off </VirtualHost> == See also == * [https://github.com/roundcube/roundcubemail/wiki Roundcube documentation] * [https://help.ubuntu.com/community/Roundcube Installing Roundcube on Ubuntu] [[Category:Server setup]] 19a698bb04ef6c85254803e8ef01515303bedd56 2860 2859 2017-01-22T11:40:06Z Neil 2 /* Installing plugins */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I'm using [https://roundcube.net/ Roundcube] as a webmail interface. * Install the packages: root@server:~# aptitude install roundcube roundcube-pgsql :(Install other <code>aspell-</code> dictionaries as you wish at the same time.) * Follow the defaults in the questions asked during installation. However, these questions don't address all the configuration options. Therefore, after installation, reconfigure the packages: root@server:~# sudo dpkg-reconfigure roundcube-core * Roundcube wants to make itself available as <code>domain.tld/roundcube</code> on all sites. The file <code>/etc/apache2/conf-available/roundcube.conf</code> does this with the <code>Alias /roundcube /var/lib/roundcube</code> directive, which is initially commented out. Keep it commented out, and copy the contents of that file into the <code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>, uncommenting the <code>Alias</code> line. * Disable the Roundcube configuration, enable the webmail site, and reload Apache: root@server:~# a2disconf roundcube root@server:~# a2ensite webmail.domain.tld.conf root@server:~# systemctl reload apache2.service * Visit <code><nowiki>https://webmail.domain.tld</nowiki></code> and you should be able to log on. ==Installing plugins== Where possible, use the [https://plugins.roundcube.net/ Roundcube plugins installer], working from the <code>/var/lib/roundcube</code> directory. ===Mobile skin=== Roundcube doesn't work well on mobile devices. The [https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile Melanie2 skin] seems to be a good alternative. The [https://roundcubeinbox.wordpress.com/2016/04/26/roundcube-for-mobile-devices/ instructions at the Roundcube inbox] are slightly unclear with paths, so here's what I did. root@server:~# cd /var/lib/roundcube root@server:/var/lib/roundcube# git clone https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile.git skins/melanie2_larry_mobile root@server:/var/lib/roundcube# php composer.phar require melanie2/mobile:dev-master :remembering to say "no" to activating the <code>jquery_mobile</code> plugin, but "yes" to activating the <code>mobile</code> plugin. ===Enigma=== The Enigma plugin comes as part of the Roundcube installation. To enable it, add <code>enigma</code> to the list of enabled plugins in <code>/etc/roundcube/config.inc.php</code>. (Note that this doesn't seem to work at the moment.) ==<code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>== <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On # SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/fullchain.pem DocumentRoot /var/lib/roundcube ServerName webmail.domain.tld <Directory /> Options +FollowSymLinks -Indexes AllowOverride None </Directory> <Directory /var/lib/roundcube/> Options +FollowSymLinks # This is needed to parse /var/lib/roundcube/.htaccess. See its # content before setting AllowOverride to None. AllowOverride All <IfVersion >= 2.3> Require all granted </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all </IfVersion> </Directory> # Protecting basic directories: <Directory /var/lib/roundcube/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/lib/roundcube/temp> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> <Directory /var/lib/roundcube/logs> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/webmail.domain.tld.access.log combined ServerSignature Off </VirtualHost> == See also == * [https://github.com/roundcube/roundcubemail/wiki Roundcube documentation] * [https://help.ubuntu.com/community/Roundcube Installing Roundcube on Ubuntu] [[Category:Server setup]] 559d791f0198d32d46ed3544b15a6abae05d1bf4 2861 2860 2017-01-22T11:53:10Z Neil 2 /* Installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I'm using [https://roundcube.net/ Roundcube] as a webmail interface. * Install the packages: root@server:~# aptitude install roundcube roundcube-pgsql roundcube-plugins :(Install other <code>aspell-</code> dictionaries as you wish at the same time.) * Follow the defaults in the questions asked during installation. However, these questions don't address all the configuration options. Therefore, after installation, reconfigure the packages: root@server:~# sudo dpkg-reconfigure roundcube-core * Roundcube wants to make itself available as <code>domain.tld/roundcube</code> on all sites. The file <code>/etc/apache2/conf-available/roundcube.conf</code> does this with the <code>Alias /roundcube /var/lib/roundcube</code> directive, which is initially commented out. Keep it commented out, and copy the contents of that file into the <code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>, uncommenting the <code>Alias</code> line. * Disable the Roundcube configuration, enable the webmail site, and reload Apache: root@server:~# a2disconf roundcube root@server:~# a2ensite webmail.domain.tld.conf root@server:~# systemctl reload apache2.service * Visit <code><nowiki>https://webmail.domain.tld</nowiki></code> and you should be able to log on. ==Installing plugins== Where possible, use the [https://plugins.roundcube.net/ Roundcube plugins installer], working from the <code>/var/lib/roundcube</code> directory. ===Mobile skin=== Roundcube doesn't work well on mobile devices. The [https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile Melanie2 skin] seems to be a good alternative. The [https://roundcubeinbox.wordpress.com/2016/04/26/roundcube-for-mobile-devices/ instructions at the Roundcube inbox] are slightly unclear with paths, so here's what I did. root@server:~# cd /var/lib/roundcube root@server:/var/lib/roundcube# git clone https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile.git skins/melanie2_larry_mobile root@server:/var/lib/roundcube# php composer.phar require melanie2/mobile:dev-master :remembering to say "no" to activating the <code>jquery_mobile</code> plugin, but "yes" to activating the <code>mobile</code> plugin. ===Enigma=== The Enigma plugin comes as part of the Roundcube installation. To enable it, add <code>enigma</code> to the list of enabled plugins in <code>/etc/roundcube/config.inc.php</code>. (Note that this doesn't seem to work at the moment.) ==<code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>== <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On # SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/fullchain.pem DocumentRoot /var/lib/roundcube ServerName webmail.domain.tld <Directory /> Options +FollowSymLinks -Indexes AllowOverride None </Directory> <Directory /var/lib/roundcube/> Options +FollowSymLinks # This is needed to parse /var/lib/roundcube/.htaccess. See its # content before setting AllowOverride to None. AllowOverride All <IfVersion >= 2.3> Require all granted </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all </IfVersion> </Directory> # Protecting basic directories: <Directory /var/lib/roundcube/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/lib/roundcube/temp> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> <Directory /var/lib/roundcube/logs> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/webmail.domain.tld.access.log combined ServerSignature Off </VirtualHost> == See also == * [https://github.com/roundcube/roundcubemail/wiki Roundcube documentation] * [https://help.ubuntu.com/community/Roundcube Installing Roundcube on Ubuntu] [[Category:Server setup]] 16ae129c0cb010b1f08727c9d1c54324429dfe18 0 1526 2862 2792 2017-02-13T11:36:38Z Neil 2 Added IRuby section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --with-library /home/user/path/to/calibre-library --auto-reload --port 8080 [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service <s>root@desktop:~# ln -s /lib/systemd/system/calibre-server.service /etc/systemd/system/calibre-server.service</s> root@desktop:~# systemctl daemon-reload <s>root@desktop:~# service calibre-server start</s> root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] 368c3d29724b7c834c5e156edf04ca32053d2d47 2870 2862 2017-06-20T08:28:35Z Neil 2 /* Create systemd job */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --auto-reload --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service <s>root@desktop:~# ln -s /lib/systemd/system/calibre-server.service /etc/systemd/system/calibre-server.service</s> root@desktop:~# systemctl daemon-reload <s>root@desktop:~# service calibre-server start</s> root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] 697bd4d2050789893bd365818e904e5ecb87319e 2871 2870 2017-06-20T08:29:12Z Neil 2 /* Create systemd job */ Updated for v3 of Calibre wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --auto-reload --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] b46648ef99a760ef5fff524c719cfc5af17e8631 2872 2871 2017-07-23T16:39:48Z Neil 2 /* Create systemd job */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] ca942d328527b19ce0acca0bd7f02a14cd003e68 2879 2872 2018-02-14T16:31:44Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] 037b25f9e4db5865e1c368cc9fb3b1c03e6b95a8 2880 2879 2018-02-14T16:42:16Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system --symlink user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] 675fd346d2abfdc3350ad935e6d4d583b677f7e1 2881 2880 2018-02-15T17:26:03Z Neil 2 /* Install Python3 and associated packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backup]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] 54cdb8056ea5a7d73faabfcb7b0616deaae3e9c5 2887 2881 2018-03-03T18:17:29Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] 817ff38f771bdf80511f75503bf7a7e80fb7b1d7 2904 2887 2018-03-16T15:13:21Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) [[Category:Server setup]] d709016825bedffc73e8341ed914db35e01dafda 0 1543 2863 2804 2017-02-27T11:03:31Z Neil 2 Added APCUPS rule wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} I used to do [[#Server (Old)|firewall setup manually]] for the server, but <code>ufw</code> has now improved to the point where I use that. See historic versions of this page from before December 2016 for details. ==Set up ufw== Enable ufw and add rules for the services the server machine will provide: root@server:~# ufw enable root@server:~# ufw allow ssh root@server:~# ufw allow http root@server:~# ufw allow https root@server:~# ufw allow domain root@server:~# ufw allow ipp root@server:~# ufw allow imap root@server:~# ufw allow smtp root@server:~# ufw allow submission root@server:~# ufw allow from 192.168.1.0/24 to any port mysql root@server:~# ufw allow from 192.168.1.0/24 to any port postgresql root@server:~# ufw allow git root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 27017:27019 # Mongo root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 28017 # Mongo web interface Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 3551 (The last rules are for [https://community.kde.org/KDEConnect KDE Connect], [https://github.com/MatejVancik/amaroKontrol amaroKontrol], and [https://help.ubuntu.com/community/apcupsd APC UPS] respectively). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 1714:1764/tcp ALLOW IN 192.168.1.0/24 8484/tcp ALLOW IN 192.168.1.0/24 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) [[Category:Server setup]] 489e6ece92eb67f9e1ac388322061ab903fb02a8 0 1515 2864 2843 2017-02-27T19:42:53Z Neil 2 wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 12.04 LTS and 13.10 (adapted from 8.04 LTS by way of 10.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Roundcube)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Desktop mail forwarding]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] * [[UPS setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] c29f9b8c01e43aeb007ca0a728ce1a72063fdc64 2885 2864 2018-03-03T17:16:53Z Neil 2 Added Discourse setup link wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 12.04 LTS and 13.10 (adapted from 8.04 LTS by way of 10.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Roundcube)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Discourse setup|Discourse forum setup]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Desktop mail forwarding]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] * [[UPS setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 020b7e3a50c60c411c0707fe1f18209eec4a820a 2901 2885 2018-03-16T12:57:02Z Neil 2 Added link to Ghost wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 12.04 LTS and 13.10 (adapted from 8.04 LTS by way of 10.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[NTP client setup]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Roundcube)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Discourse setup|Discourse forum setup]] * [[Ghost setup|Ghost blog setup]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[NTP client setup]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Desktop mail forwarding]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] * [[UPS setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] 5dd31c5600d1f5123979aba991d253acbb388743 0 1648 2865 2017-02-27T19:51:42Z Neil 2 Created page with "I have an APC-made UPS (uninterruptable power supply). It comes with the ability for connected PCs to monitor its state. I have it connected by USB cable to the desktop machi..." wikitext text/x-wiki I have an APC-made UPS (uninterruptable power supply). It comes with the ability for connected PCs to monitor its state. I have it connected by USB cable to the desktop machine. That machine communicates with the server, so they are both aware of power status. ==Installation== Simply follow the [https://help.ubuntu.com/community/apcupsd Ubuntu instructions]. The desktop machine has <code>/etc/apcupsd/apcupsd.conf</code> containing: UPSCABLE usb UPSTYPE usb DEVICE POLLTIME 60 The server machine has <code>/etc/apcupsd/apcupsd.conf</code> UPSCABLE ether UPSTYPE net DEVICE desktop.domain.tld:3551 POLLTIME 10 The desktop machine will require port 3551 open on its [[Firewall setup|firewall]]. On both machines, set <code>ISCONFIGURED=yes</code> in <code>/etc/default/apcupsd</code> and restart the monitor. root@server:~# systemctl restart apcupsd.service root@server:~# apcaccess status == See also == * [https://help.ubuntu.com/community/apcupsd Ubuntu instructions on apcupsd] * [http://www.apcupsd.org/manual/manual.html#nis-server-client-configuration-using-the-net-driver acpupsd documentation]. [[Category:Server setup]] 2390e0d3617f5e909c0a1f0ee5722a3bcbdda107 2866 2865 2017-02-27T19:53:28Z Neil 2 wikitext text/x-wiki I have an APC-made UPS (uninterruptable power supply). It comes with the ability for connected PCs to monitor its state. I have it connected by USB cable to the desktop machine. That machine communicates with the server, so they are both aware of power status. ==Installation== Simply follow the [https://help.ubuntu.com/community/apcupsd Ubuntu instructions]. The desktop machine has <code>/etc/apcupsd/apcupsd.conf</code> containing: UPSCABLE usb UPSTYPE usb DEVICE POLLTIME 60 The server machine has <code>/etc/apcupsd/apcupsd.conf</code> UPSCABLE ether UPSTYPE net DEVICE desktop.domain.tld:3551 POLLTIME 10 The desktop machine will require port 3551 open on its [[Firewall setup|firewall]]. On both machines, set <code>ISCONFIGURED=yes</code> in <code>/etc/default/apcupsd</code> and restart the monitor. root@server:~# systemctl restart apcupsd.service root@server:~# apcaccess status Finally, configure the BIOS on all machines to start automatically when power is supplied. == See also == * [https://help.ubuntu.com/community/apcupsd Ubuntu instructions on apcupsd] * [http://www.apcupsd.org/manual/manual.html#nis-server-client-configuration-using-the-net-driver acpupsd documentation]. [[Category:Server setup]] f819df4919a77f1cd16bb30c8001b51bc16b5461 0 1531 2867 2853 2017-03-02T11:56:04Z Neil 2 /* SpamAssassin */ Added auto-respawn instructions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL.) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Sending Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 8537942aed22d375e6a11adb6bf2828c38ed0f6d 2874 2867 2017-11-01T16:28:40Z Neil 2 Fixed updated link wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = . # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL.) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 07fa0a9d4f853cac5168a51660ade955e428f0e1 2894 2874 2018-03-04T10:16:53Z Neil 2 /* Basic installation */ Updated recipient_delimiter wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL.) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] fd96560531e2554d0c02b1c52a001f69bbd033c4 0 1533 2868 2828 2017-04-12T13:55:25Z Neil 2 Updated commands for systemd wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# systemctl reload apache2.service ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# ssystemctl reload apache2.service == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# systemctl reload apache2.service == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/cert.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# systemctl restart apache2.service Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ===Note to self=== On Apache version 2.4.8 and above, replace: SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem with: SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off ==Add modules needed== root@server:~# a2enmod headers root@server:~# a2enmod cgi root@server:~# systemctl restart apache2.service == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] daf52c39163cb1ed08d58c2100856dd6ac674e05 0 1536 2869 2169 2017-05-19T13:07:41Z Neil 2 Updated min samba protocol requirement wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Git setup|Git]] |align="center" width="50%"| [[Log viewer setup|Log stats]] |} Samba allows me to grant access to Linux shares from Windows machines. This setup is done on the desktop machine. * Install the packages you need root@desktop:~# apt-get install samba * Modify <code>/etc/samba/smb.conf</code> (note that you'll have to skip around the file to find all the settings) [global] min protocol = SMB2 ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of workgroup = myworkgroup ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. security = user username map = /etc/samba/smbusers # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. unix password sync = no #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home directory as \\server\username [homes] comment = Home Directories browseable = yes # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. read only = no # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # This might need tweaking when using external authentication schemes valid users = %S * For each user, create the password for that user root@desktop:~# smbpasswd -a linuxusername * Add the user entries into <tt>/etc/samba/smbusers</tt> linuxusername1 = "sambausername1" linuxusername2 = "sambausername2" :Note that the quotes are important. This allows someone to log in using sambausername1 and gain access to linuxuername1's home directory on the Ubuntu desktop machine * Restart Samba: root@desktop:~# /etc/init.d/samba restart Note that each user will need to log into the Samba server machine directly (i.e. via the GUI at that machine) to go through the 'first login mandatory password reset' rigmarole. Once that's done, they'll be able to use their Samba share. On the Windows side, the user should point Windows Explorer at <code>\\server\username</code> and give the Samba username and password to log in. They can then map a new network drive to the Samba directory if they want. == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.howtogeek.com/howto/ubuntu/install-samba-server-on-ubuntu/ Install Samba Server on Ubuntu], from the Howto Geek. * [http://www.howtogeek.com/howto/ubuntu/share-ubuntu-home-directories-using-samba/ Share Ubuntu Home Directories using Samba], again from the Howto Geek. * [http://us6.samba.org/samba/docs/Samba24Hc13.pdf Samba troubleshooting guide] (pdf) [[Category:Server setup]] 6d7b7da0452d18b5a19122415d813c85c0a98322 0 1534 2873 2851 2017-10-29T12:26:17Z Neil 2 /* Tweaking Mediawiki */ Added comments on mobile front end wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir /tmp/site.domain.tld/ root@server:~# chown www-data:www-data /tmp/site.domain.tld/ * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/tmp/domain.tld"; That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] a971e316cb0498de01d6c8ba781198e8c364a26e 0 1593 2875 2836 2017-11-30T22:37:31Z Neil 2 /* Common tasks */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Scanner setup|Scanner]] |align="center" width="50%"| [[Samba setup|Samba]] |} Git is a distributed version control system. I use Gitolite as a centralised git server. While the repositories reside on the server, administration is done by <code>user@desktop</code> on the desktop machine. Gitweb is used to give read-only access to some, but not all, repositories. Gitweb will only show repos that have an <code>owner</code>, <code>desc</code>, or <code>category</code> line in <code>gitolite.conf</code>; other repos will be hidden from Gitweb. ==Install Git and Gitolite== Unfortunately, only Gitolite v2 is in the Ubuntu repositories, so I have to download Gitolite from source. * Install what you can from the Ubuntu repos root@server:~# aptitude install git git-doc gitweb highlight * Create the <code>git</code> user: root@server:~# adduser --disabled-password git root@server:~# su - git git@server:~$ mkdir -p bin git@server:~$ logout :(Logout then log back in again so that the <code>.profile</code> script adds <code>~/bin</code> to the <code>PATH</code> environment variable.) * Add git to the list of users allowed to SSH into the server. Modify <code>/etc/ssh/sshd_config</code> so the <code>AllowUsers</code> line includes git: # Only allow access by certain users AllowUsers user1 user2 git :then restart sshd: root@server:~# service ssh restart * Back on the '''desktop''' machine, create a public RSA key for user@desktop (the administrator user Gitolite): user@desktop:~# ssh-keygen -t rsa * On the '''server''', copy the key to the git user's home: root@server:~# scp user@desktop:~/.ssh/id_rsa.pub /home/git/user.pub * Download and install Gitolite root@server:~# su - git git@server:~$ git clone git://github.com/sitaramc/gitolite git@server:~$ gitolite/install -to bin git@server:~$ gitolite setup -pk user.pub * On the '''desktop''' machine, clone and modify the <code>gitolite-admin</code> repo as you see fit. user@desktop:~$ git clone git@server:gitolite-admin user@desktop:~$ cd gitolite ==Install and setup git-daemon== Git-daemon allows low-resource anonymous cloning of repositories * Install <code>git-daemon</code> root@server:~# aptitude install git-daemon-run * Modify <code>/etc/sv/git-daemon/run</code>, modifying the user group and the paths (the last two lines): #!/bin/sh exec 2>&1 echo 'git-daemon starting.' exec chpst -ugitdaemon:git \ "$(git --exec-path)"/git-daemon --verbose \ --base-path=/home/git/repositories/ /home/git/repositories/ * Restart git-daemon: root@server:~# sv restart git-daemon * For each repo you want accessible by git-daemon, add the <code>R = daemon</code> to the repo's config in <code>gitolite.conf</code> repo testing RW+ = @all R = daemon :then commit and push the changed admin file. ==Setup gitweb== * Modify the <code>UMASK</code> setting (to allow gitweb to read the repos) and the <code>GIT_CONFIG_KEYS</code> setting (to allow Gitolite to set gitweb descriptions) in <code>/home/git/.gitolite.rc</code>: UMASK => 0027, GIT_CONFIG_KEYS => 'gitweb\..*', * Modify <code>/etc/gitweb.conf</code> to include: $projectroot = "/home/git/repositories"; $projects_list = "/home/git/projects.list"; # Allow syntax highlighting $feature{'highlight'}{'default'} = [1]; * Add Apache's user to the git group: root@server:~# usermod -a -G git www-data * Check the contents of <code>/etc/apache2/conf-available/gitweb.conf</code>: Alias /gitweb /usr/share/gitweb <Directory /usr/share/gitweb> Options +FollowSymLinks +ExecCGI AddHandler cgi-script .cgi </Directory> :They should be OK without modification. There's no need to update <code>/etc/apache2/sites-available/site.domain.tld</code>. * Enable the gitweb config and reload Apache: root@server:~# a2enconf gitweb root@server:~# service apache2 reload * You should be able to view your repos from <code><nowiki>http://site.domain.tld/gitweb/</nowiki></code> ===Gitweb in its own virtual host=== Create the file <code>/etc/apache2/sites-available/git.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /usr/share/gitweb ServerName git.domain.tld ServerAlias git Alias /gitweb /usr/share/gitweb # Defined in /etc/apache2/conf.d/gitweb # <Directory /usr/share/gitweb> # Options FollowSymLinks +ExecCGI # AddHandler cgi-script .cgi # </Directory> RewriteEngine on # make the front page an internal rewrite to the gitweb script RewriteRule ^/$ /gitweb.cgi # make access for "dumb clients" work RewriteRule ^/(.*\.git/(?!/?(HEAD|info|objects|refs)).*)?$ \ /gitweb.cgi%{REQUEST_URI} [L,PT] # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/git.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature Off </VirtualHost> * Enable the site: root@server:~# a2ensite git.domain.tld root@server:~# service apache2 reload ==Changing the default template== Modify the standard template if you want changes to affect all new repos created. Do this, for instance, if you want to use [https://keybase.io/neilnjae Keybase.io] to automatically sign all commits by adding the file <code>hooks/pre-commit</code>: #!/bin/sh # Run Keybase to sign the commit keybase dir sign -p git git add SIGNED.md ===Server side=== Modify the contents of <code>/usr/share/git-core/templates/</code> as you see fit, such as adding the commit hook above. This should affect all repos created with Gitolite. ===Client side=== Copy <code>/usr/share/git-core/templates/</code> to somewhere local, such as <code>~/.git_template</code>. Modify the contents as you see fit. Tell Git to use this template rather than the default one: user@desktop:~$ git config --global init.templatedir '~/.git_template' This should affect all new repos created on the client, outside of Gitolote. ==Import prior repos== * Copy them to <code>/home/git/repositories</code> * Make them git-writable and gitweb-readable: root@server:~# chown -R git:git /home/git/repositories root@server:~# chmod -R g+rX ~/repositories/ * On the desktop, modify <code>gitolite-admin/conf/gitolite.conf</code> repo repo1 RW+ = user1 * Manually add any existing gitweb descriptions to the desktop's <code>gitolite-admin/conf/gitolite.conf</code> file. (You can find the descriptions in <code>/home/git/repositories/existing-repo.git/description</code> * Commit and push the changes user@desktop:~/gitolite-admin$ git commit -a -m "Added imported repos" user@desktop:~/gitolite-admin$ git push ==Common tasks== * To clone a repo: git clone git@git.domain.tld:repo.git * Create a new repo, first create its entry in <code>gitolite-admin/conf/gitolite.conf</code> and push the changes. This will create an empty repo on the server. Then clone it on the client machine, ignoring the message about cloning an empty repo. * To import a repo into Github, use the Github "import repository" functionality, using the url <code><nowiki>git://git.domain.tld/repo.git</nowiki></code> . Note that you may need to remove files larger than 100Mb, following [https://help.github.com/articles/remove-sensitive-data/ Github's instructions]. * To add Github as a remote for a repo, git remote add github git@github.com:neilnjae/foo.git git push github master :(Use this technique to import a repo on Github if the import doesn't work: create an empty repo on Github, add it as another remote for the existing repo, then push the changes to Github.) ==See also== Gitolite 3 docs: * [http://sitaramc.github.com/gitolite/qi.html Gitolite quick install guide] * [http://sitaramc.github.com/gitolite/users.html Adding users in Gitolite] * [http://sitaramc.github.com/gitolite/repos.html Creating new repos in Gitolite] * [http://sitaramc.github.com/gitolite/rare.html#existing Importing existing repos into Gitolite] Installation walkthroughs: * [http://silas.sewell.org/blog/2011/01/08/setup-gitolite-on-ubuntu/ An install walkthrough] * [http://computercamp.cdwilson.us/git-gitolite-git-daemon-gitweb-setup-on-ubunt A good walkthorugh, but with superfluous steps] [[Category:Server setup]] 3ddcc63b138d314b0e7390b2367f28b09884f3c5 0 1541 2876 2698 2018-02-02T16:56:15Z Neil 2 /* Static IP numbers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/resolv.conf</code> to include domain domain.tld search domain.tld :(you may need to include these lines in <code>/etc/resolvconf/resolv.conf.d/tail</code> if <code>resolvconf</code> keeps overwriting <code>/etc/resolv.conf</code>) * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * To give a static IP to a WiFi connection using WPA Supplicant, use this interface stanza: iface wlan0 inet static address 192.168.1.101 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 wpa-essid MyWifiID wpa-psk my-secret-key # wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf # iface default inet dhcp * Ensure that <code>/etc/resolv.conf</code> is a symbolic link to <code>/run/resolvconf/resolv.conf</run>. If it's still pointing to somewhere with <code>NetworkManager</code> in the name, delete it and recreate it: root@desktop:~# rm /etc/resolv.conf root@desktop:~# ln -s /run/resolvconf/resolv.conf /etc/resolv.conf (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] 60a24907d473a2a264f3dfa259fc7e1f5404e1d8 2877 2876 2018-02-02T16:57:05Z Neil 2 /* Hostname */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * To give a static IP to a WiFi connection using WPA Supplicant, use this interface stanza: iface wlan0 inet static address 192.168.1.101 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 wpa-essid MyWifiID wpa-psk my-secret-key # wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf # iface default inet dhcp * Ensure that <code>/etc/resolv.conf</code> is a symbolic link to <code>/run/resolvconf/resolv.conf</run>. If it's still pointing to somewhere with <code>NetworkManager</code> in the name, delete it and recreate it: root@desktop:~# rm /etc/resolv.conf root@desktop:~# ln -s /run/resolvconf/resolv.conf /etc/resolv.conf (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] 7d4b970cb8bf5c6be92a987b3be195406a3b9071 2878 2877 2018-02-02T17:00:30Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Disable NetworkManager == Change <code>/etc/NetworkManager/NetworkManager.conf</code> so that <code>managed=true</code> and there's no <code>dns=dnsmasq</code> line active: [main] plugins=ifupdown,keyfile #dns=dnsmasq [ifupdown] managed=true == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * To give a static IP to a WiFi connection using WPA Supplicant, use this interface stanza: iface wlan0 inet static address 192.168.1.101 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 wpa-essid MyWifiID wpa-psk my-secret-key # wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf # iface default inet dhcp * Ensure that <code>/etc/resolv.conf</code> is a symbolic link to <code>/run/resolvconf/resolv.conf</run>. If it's still pointing to somewhere with <code>NetworkManager</code> in the name, delete it and recreate it: root@desktop:~# rm /etc/resolv.conf root@desktop:~# ln -s /run/resolvconf/resolv.conf /etc/resolv.conf (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] d750b0a597c124a88767ec705cee7947e07094d8 0 1532 2882 2803 2018-02-27T11:54:58Z Neil 2 Changed references from Squirrelmail to Roundcube wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.roundcube.net/ Roundcube] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Roundcube setup is described in the [[Webmail setup]] page, but requires the [[Web server setup]], including HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap ==== Note to self: don't disable to protocol, turn them off in 10-master.conf ==== * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. ==== Note to self: also disable pop3, as below: ==== service pop3-login { inet_listener pop3 { #port = 110 port = 0 } inet_listener pop3s { #port = 995 #ssl = yes port = 0 } } * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl = yes ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth test user@domain.tld mailpassword root@server:~# doveadm auth login user@domain.tld mailpassword * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 707667139495a426012eec9e2355bb08af2c0917 0 1645 2883 2856 2018-02-28T10:42:12Z Neil 2 Updated to refer to certbot wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.io/en/latest/install.html install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== This is a bit of chicken-and-egg. First, you need to create the non-HTTPS virtual sites for all the domains you want to serve. You can then use the Apache plugin to the LetsEncrypt client to fetch and install the certificates. Then, you can go back and fix the various certificate settings in the virtual host conf files. If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# certbot --apache -d domain.tld -d www.domain.tld root@server:# certbot --apache -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# certbot --apache -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--cert-name</code> option and list ''all'' the domains for the certificate: root@server:~# certbot --cert-name domain.tld -d www.domain.tld,other.domain.tld root@server:~# systemctl reload apache2.service Domains not listed in the `certbot --cert-name` command will be removed from the certificate. ==Check the contents of a certificate== If you want to see what certificates you have, use `certbot`: root@server:~# certbot certificates If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==Automatically update certificates== Create the file <code>/etc/cron.daily/letsencrypt-renew</code>: #!/bin/sh /usr/bin/certbot renew systemctl reload apache2.service systemctl reload dovecot.service systemctl reload postfix.service Make it executable: root@server:# chmod +x /etc/cron.daily/letsencrypt-renew ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] fb2000dfe9aeb1aea8aa4939e645f52ea783a0ac 2884 2883 2018-03-01T11:23:36Z Neil 2 /* Extend the domains of a certificate */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://letsencrypt.readthedocs.io/en/latest/install.html install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== This is a bit of chicken-and-egg. First, you need to create the non-HTTPS virtual sites for all the domains you want to serve. You can then use the Apache plugin to the LetsEncrypt client to fetch and install the certificates. Then, you can go back and fix the various certificate settings in the virtual host conf files. If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# certbot --apache -d domain.tld -d www.domain.tld root@server:# certbot --apache -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# certbot --apache -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--cert-name</code> option and list ''all'' the domains for the certificate: root@server:~# certbot certonly --cert-name domain.tld -d domain.tld,www.domain.tld,other.domain.tld root@server:~# systemctl reload apache2.service Domains not listed in the <code>certbot --cert-name</code> command will be removed from the certificate. Note that you need to include the original <code>domain.tld</code> certificate name in the certificate expansion command. Luckily, <code>certbot</code> asks you to confirm changes before you make them. ==Check the contents of a certificate== If you want to see what certificates you have, use `certbot`: root@server:~# certbot certificates If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==Automatically update certificates== Create the file <code>/etc/cron.daily/letsencrypt-renew</code>: #!/bin/sh /usr/bin/certbot renew systemctl reload apache2.service systemctl reload dovecot.service systemctl reload postfix.service Make it executable: root@server:# chmod +x /etc/cron.daily/letsencrypt-renew ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] 9426ba8be38b352192fd5a42aa93e1a5d1217121 0 1649 2886 2018-03-03T18:16:32Z Neil 2 Created page with "{|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''Server setup..." wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Users to log in via Google, Twitter, GitHub, or Facebook? ==Reply by email== Users to post replies via email? Configure reply via email. Email reply pattern ==Backups== Automatic daily backups? Configure backups. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] 0fdb712d3c0ea288b0b710570b50e7ef0b7c5ad9 2889 2886 2018-03-03T18:56:43Z Neil 2 /* Social logins */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Users to post replies via email? Configure reply via email. Email reply pattern ==Backups== Automatic daily backups? Configure backups. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] abf9cb2eb5ab96ad69db570f0426f9e872da4b53 2890 2889 2018-03-03T20:29:44Z Neil 2 /* Reply by email */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Users to post replies via email? Configure reply via email. Email reply pattern forum.domain.tld/admin/site_settings/category/email reply by email address is replies.%{reply_key}@domain.tld ==Backups== Automatic daily backups? Configure backups. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] 44d0a3ab607ecdd77e4626ea509452e0345490c6 2891 2890 2018-03-04T09:55:10Z Neil 2 /* Email accounts */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [#Reply_by_email|receive email replies]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Users to post replies via email? Configure reply via email. Email reply pattern forum.domain.tld/admin/site_settings/category/email reply by email address is replies.%{reply_key}@domain.tld ==Backups== Automatic daily backups? Configure backups. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] a296fa28152d99745963528bbba1526533e73b1c 2892 2891 2018-03-04T09:55:35Z Neil 2 /* Email accounts */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Users to post replies via email? Configure reply via email. Email reply pattern forum.domain.tld/admin/site_settings/category/email reply by email address is replies.%{reply_key}@domain.tld ==Backups== Automatic daily backups? Configure backups. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] d9e147c505e560416ae4753e0c417be823fd79e9 2893 2892 2018-03-04T09:56:34Z Neil 2 /* Social logins */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with all four APIs in the ''Social'' category: Contacts, People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Users to post replies via email? Configure reply via email. Email reply pattern forum.domain.tld/admin/site_settings/category/email reply by email address is replies.%{reply_key}@domain.tld ==Backups== Automatic daily backups? Configure backups. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] f91e998b0ff23feef3716d34567a9e7554e323e9 2895 2893 2018-03-04T11:06:50Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with all four APIs in the ''Social'' category: Contacts, People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Again, the "[https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email]" instructions are pretty good. I changed the <code>reply by email address</code> to be <code>replies.%{reply_key}@domain.tld</code>, with a <code>.</code> rather than a <code>+</code>, because I set up Postfix to use <code>.</code> as the <code>recipient_delimiter</code> You can set up categories to have threads created by email. In the category settings dialog, under the "Settings" tab, create a custom email address for that category, such as <code>mycat@domain.tld</code> for the <code>My Category</code> category. Then, in <code>/etc/postfix/valiases</code>, set the alias: mycat@domain.tld replies.mycat@domain.tld and refresh the map root@server:/etc/postfix# postmap valiases This will forward messages to <code>mycat@domain.tld</code> into the <code>My Category</code> category, so long as they're sent from registered users. Users to post replies via email? Configure reply via email. Email reply pattern forum.domain.tld/admin/site_settings/category/email reply by email address is replies.%{reply_key}@domain.tld ==Backups== Automatic daily backups? Configure backups. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] ac1c7e600b25c037c585c5e2b70b4150ae80754b 2896 2895 2018-03-04T11:07:13Z Neil 2 /* Reply by email */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with all four APIs in the ''Social'' category: Contacts, People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Again, the "[https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email]" instructions are pretty good. I changed the <code>reply by email address</code> to be <code>replies.%{reply_key}@domain.tld</code>, with a <code>.</code> rather than a <code>+</code>, because I set up Postfix to use <code>.</code> as the <code>recipient_delimiter</code> You can set up categories to have threads created by email. In the category settings dialog, under the "Settings" tab, create a custom email address for that category, such as <code>mycat@domain.tld</code> for the <code>My Category</code> category. Then, in <code>/etc/postfix/valiases</code>, set the alias: mycat@domain.tld replies.mycat@domain.tld and refresh the map root@server:/etc/postfix# postmap valiases This will forward messages to <code>mycat@domain.tld</code> into the <code>My Category</code> category, so long as they're sent from registered users. ==Backups== Automatic daily backups? Configure backups. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] d20807b7348402a407b07a3cdc45f540ddabd2b7 2897 2896 2018-03-04T14:37:32Z Neil 2 /* Backups */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with all four APIs in the ''Social'' category: Contacts, People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Again, the "[https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email]" instructions are pretty good. I changed the <code>reply by email address</code> to be <code>replies.%{reply_key}@domain.tld</code>, with a <code>.</code> rather than a <code>+</code>, because I set up Postfix to use <code>.</code> as the <code>recipient_delimiter</code> You can set up categories to have threads created by email. In the category settings dialog, under the "Settings" tab, create a custom email address for that category, such as <code>mycat@domain.tld</code> for the <code>My Category</code> category. Then, in <code>/etc/postfix/valiases</code>, set the alias: mycat@domain.tld replies.mycat@domain.tld and refresh the map root@server:/etc/postfix# postmap valiases This will forward messages to <code>mycat@domain.tld</code> into the <code>My Category</code> category, so long as they're sent from registered users. ==Backups== Enable [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 daily backups] in Discourse, then ensure that files in <code>/var/discourse/shared/standalone/backups/default/*</code> are [[https://www.njae.me.uk/Backup_setup backed up daily]]. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] 472287683e4db7566b16128f631cc8d728d7edc3 2898 2897 2018-03-04T14:40:22Z Neil 2 /* Reply by email */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with all four APIs in the ''Social'' category: Contacts, People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Again, the "[https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email]" instructions are pretty good. I set the <code>reply by email address</code> to be <code>replies+%{reply_key}@domain.tld</code>. Make sure the character after <code>replies</code> matches one of the characters specified as a <code>recipient_delimiter</code> in [[Postfix server setup|Postfix]]. You can set up categories to have threads created by email. In the category settings dialog, under the "Settings" tab, create a custom email address for that category, such as <code>mycat@domain.tld</code> for the <code>My Category</code> category. Then, in <code>/etc/postfix/valiases</code>, set the alias: mycat@domain.tld replies.mycat@domain.tld and refresh the map root@server:/etc/postfix# postmap valiases This will forward messages to <code>mycat@domain.tld</code> into the <code>My Category</code> category, so long as they're sent from registered users. ==Backups== Enable [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 daily backups] in Discourse, then ensure that files in <code>/var/discourse/shared/standalone/backups/default/*</code> are [[https://www.njae.me.uk/Backup_setup backed up daily]]. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] c973e34e76dc74eedb18dbbf298c366991d69a9b 2899 2898 2018-03-04T15:09:01Z Neil 2 /* Apache proxy */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/forum.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with all four APIs in the ''Social'' category: Contacts, People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Again, the "[https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email]" instructions are pretty good. I set the <code>reply by email address</code> to be <code>replies+%{reply_key}@domain.tld</code>. Make sure the character after <code>replies</code> matches one of the characters specified as a <code>recipient_delimiter</code> in [[Postfix server setup|Postfix]]. You can set up categories to have threads created by email. In the category settings dialog, under the "Settings" tab, create a custom email address for that category, such as <code>mycat@domain.tld</code> for the <code>My Category</code> category. Then, in <code>/etc/postfix/valiases</code>, set the alias: mycat@domain.tld replies.mycat@domain.tld and refresh the map root@server:/etc/postfix# postmap valiases This will forward messages to <code>mycat@domain.tld</code> into the <code>My Category</code> category, so long as they're sent from registered users. ==Backups== Enable [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 daily backups] in Discourse, then ensure that files in <code>/var/discourse/shared/standalone/backups/default/*</code> are [[https://www.njae.me.uk/Backup_setup backed up daily]]. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] 5b7ebf0374c9cf4113172dec9f4d8d9f8db8e96b 2900 2899 2018-03-04T15:10:52Z Neil 2 /* Apache proxy */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. : Note that the main forum virtual host has a different log path from the rest of the sites on this domain. This stops [[Log_viewer_setup|the web log viewer]] getting swamped by all the internal Discourse page requests. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/forum.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with all four APIs in the ''Social'' category: Contacts, People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Again, the "[https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email]" instructions are pretty good. I set the <code>reply by email address</code> to be <code>replies+%{reply_key}@domain.tld</code>. Make sure the character after <code>replies</code> matches one of the characters specified as a <code>recipient_delimiter</code> in [[Postfix server setup|Postfix]]. You can set up categories to have threads created by email. In the category settings dialog, under the "Settings" tab, create a custom email address for that category, such as <code>mycat@domain.tld</code> for the <code>My Category</code> category. Then, in <code>/etc/postfix/valiases</code>, set the alias: mycat@domain.tld replies.mycat@domain.tld and refresh the map root@server:/etc/postfix# postmap valiases This will forward messages to <code>mycat@domain.tld</code> into the <code>My Category</code> category, so long as they're sent from registered users. ==Backups== Enable [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 daily backups] in Discourse, then ensure that files in <code>/var/discourse/shared/standalone/backups/default/*</code> are [[https://www.njae.me.uk/Backup_setup backed up daily]]. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] 4430093b4b90cf63556716f313dd5f21044f021e 2903 2900 2018-03-16T15:13:16Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Backup setup|Backups]] |align="center" width="50%"| [[Ghost setup|Ghost]] |} [https://github.com/discourse/discourse Discourse] is an opinionated system, so you have to work around it a bit to get it to play nicely with my existing setup. Discourse runs in its own Docker container. Apache proxys requests to the appropriate URL to this container. ==Preparation== * Add any domain names you need for the Discourse forums (e.g. <code>forum.domain.tld</code>) and the POP3 server you're about to enable (e.g. <code>pop.domain.tld</code>), both to your [DNS server setup|local DNS] and an external DNS. * [[SSL certificate generation|Update any SSL certificates]] you need for the new forum domain name. ==Email accounts== Discourse uses POP3 for reading incoming email. * Enable POP3. Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to enable it, but removing any <code>port = 0</code> lines in the POP3 sections: service pop3-login { inet_listener pop3 { port = 110 # port = 0 } inet_listener pop3s { port = 995 ssl = yes # port = 0 } } * Set up the <code>forum@domain.tld</code> and <code>replies@domain.tld</code> email accounts you need for Discourse. The first is the account used by Discourse to send emails; the second is the account used to [[#Reply_by_email|receive email replies]]. root@server:~# cd /etc/dovecot/ root@server:~# dovecot-adduser forum@domain.tld mailpassword root@server:~# dovecot-adduser replies@domain.tld mailpassword root@server:~# doveadm auth test forum@domain.tld mailpassword root@server:~# doveadm auth test replies@domain.tld mailpassword * Ensure that <code>domain.tld</code> is listed in <code>/etc/postfix/vhosts</code>, and ''not'' included in any <code>virtual_alias_domains</code> statements in <code>/etc/postfix/main.cf</code> . * [[SSL certificate generation|Update your mailserver SSL certificates]] to include <code>pop.domain.tld</code>. * Open the server's firewall for POP3 access, but only for the local LAN and the Docker installation. root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 192.168.0.0/16 to any port pop3s root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3 root@server:~# ufw allow in proto tcp from 172.17.0.0/16 to any port pop3s * Reload Dovecot and Postfix for the new configuration root@server:~# systemctl reload dovecot.service root@server:~# systemctl reload postfix.service ==Install Docker community edition== This involves adding a new PPA repository rather than Ubuntu's default version. Follow the [https://docs.docker.com/install/linux/docker-ce/ubuntu/ Docker Community Edition instructions]. ==Basic installation== Discourse requires that it's the only thing listening on port 80 (and ideally port 443 as well) during setup. Therefore, you need to stop Apache before installing and configuring Discourse. * Disable Apache: root@server:~# systemctl stop apache2 root@server:~# killall apache2 :(You might need to kill leftover threads manually) * Install Discourse using the [https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md standard instructions]. At present, don't set up any SSL support: that will be handled by the main Apache2 installation. It may take some time. ==Apache proxy== We'll now set up Apache to forward requests to <code>forum.domain.tld</code> to Discourse, while still serving the remainder of the sites itself. * Stop Discourse root@server:/var/discourse# ./launcher stop app * Configure Discourse to serve port 7654 rather than 80 or 443. Edit <code>/var/discourse/containers/app.yml</code> expose: # - "80:80" # http # - "443:443" # https - "7654:80" * Enable the Apache proxy modules root@server:~# a2enmod proxy root@server:~# a2enmod proxy_http * Edit the <code>/etc/apache2/sites-available/domain.tld.conf</code> file to include the proxy information. Ensure this section comes before any wildcard matches for other subdomains on this domain. : Note that the main forum virtual host has a different log path from the rest of the sites on this domain. This stops [[Log_viewer_setup|the web log viewer]] getting swamped by all the internal Discourse page requests. <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName forum.domain.tld SSLEngine On SSLProxyEngine On SSLCertificateKeyFile /etc/letsencrypt/live/www.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/www.domain.tld/fullchain.pem SSLProxyCheckPeerName Off SSLProxyCheckPeerExpire off SSLProxyVerify none ProxyPreserveHost On ProxyRequests Off ProxyPass / <nowiki>http://0.0.0.0:7654/</nowiki> ProxyPassReverse / <nowiki>http://0.0.0.0:7654/</nowiki> CustomLog /var/log/apache2/forum.domain.tld.access.log combined ErrorLog /var/log/apache2/error.log ServerSignature off </VirtualHost> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName forum.domain.tld Redirect permanent / <nowiki>https://forum.mk-rpg.org.uk/</nowiki> CustomLog /var/log/apache2/domain.tld.access.log combined ServerSignature off </VirtualHost> * Restart Apache2: root@server:~# systemctl restart apache2 * Restart Discourse: root@server:/var/discourse# ./launcher stop app You should now be able to visit <code><nowiki>http://forum.domain.tld</nowiki></code> and see the Discourse site. ==Social logins== Essentially, follow the instructions for OAuth logins with [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Google] and [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Facebook]. The only things to watch out for are the API's Google wants authorised (I went with all four APIs in the ''Social'' category: Contacts, People, Google+, and Google+ Apps) and the OAuth redirect URIs. * For Google, the URI is <code><nowiki>http://forum.domain.tld/auth/google_oauth2/callback</nowiki></code> * For Facebook, the URI is <code><nowiki>http://forum.domain.tld/auth/facebook/callback</nowiki></code> Note the lack of <code>s</code> in the <code>http</code>. ==Reply by email== Again, the "[https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email]" instructions are pretty good. I set the <code>reply by email address</code> to be <code>replies+%{reply_key}@domain.tld</code>. Make sure the character after <code>replies</code> matches one of the characters specified as a <code>recipient_delimiter</code> in [[Postfix server setup|Postfix]]. You can set up categories to have threads created by email. In the category settings dialog, under the "Settings" tab, create a custom email address for that category, such as <code>mycat@domain.tld</code> for the <code>My Category</code> category. Then, in <code>/etc/postfix/valiases</code>, set the alias: mycat@domain.tld replies.mycat@domain.tld and refresh the map root@server:/etc/postfix# postmap valiases This will forward messages to <code>mycat@domain.tld</code> into the <code>My Category</code> category, so long as they're sent from registered users. ==Backups== Enable [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 daily backups] in Discourse, then ensure that files in <code>/var/discourse/shared/standalone/backups/default/*</code> are [[https://www.njae.me.uk/Backup_setup backed up daily]]. ==See also== * [https://meta.discourse.org/t/17247 Discourse proxy instructions] * [https://meta.discourse.org/t/configuring-google-login-for-discourse/15858 Login with Google] * [https://meta.discourse.org/t/configuring-facebook-login-for-discourse/13394 Login with Facebook] * [https://meta.discourse.org/t/set-up-reply-via-email-support-e-mail/14003 Reply by email] * [https://meta.discourse.org/t/configure-automatic-backups-for-discourse/14855 Backups] [[Category:Server setup]] 2537901629f8be39356f6ebe7d565086bfce0d73 0 1540 2888 2845 2018-03-03T18:18:02Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly [[Category:Server setup]] 2cd2ee79fc171efb2ce89213149eb3213d442461 0 1650 2902 2018-03-16T15:12:21Z Neil 2 Created page with "{|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''Server setup..." wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== * Install [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node 0.8], following the standard instructions there. * Install [https://docs.ghost.org/docs/install Ghost command-line interface] * Install [https://docs.ghost.org/docs/cli-install Ghost from the command line] I created an additional user for the installation, but I'm not sure that was necessary. The standard non-root user on the server should do. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/theme$ npm install user@desktop:~/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/theme$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip ghost-theme-willow * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-html.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. Install Disqus https://help.ghost.org/article/15-disqus https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda Site worknjae Add work.njae.me.uk as a Disqus trusted domain. Disable "Tracking" and "Affiliated sites" in Disqus advanced settings. https://worknjae.disqus.com/admin/settings/advanced/ Install MathJax Note that you have to type \\( and \\) , or \\[ \\] to embed maths. Install Prism https://bell0bytes.eu/syntax-latex/ Install theme Using Willow. https://www.ghostforbeginners.com/willow/ See below for images See /var/www/work.njae.me.uk/content/themes/ghost-theme-willow-master/partials/sidebar.hbs for sidebar changes. Note need to restart Ghost after changes. source/sass/components/_post-view.scss for image floats img[src$="#left"] { max-width:50vw; float: left; } img[src$="#right"] { max-width:50vw; float: right; } partials/sidebar.hbs post.hbs source/sass/components/_post_view zip -x " * /node_modules/ * " ' * / .git / *' -r ghost-theme-willow.zip ghost-theme-willow (but without the spaces) Others are https://www.ghostforbeginners.com/coder/ https://www.ghostforbeginners.com/promenade/ https://www.ghostforbeginners.com/ghoststa/ Search https://mrvautin.com/how-to-add-search-functionality-to-your-ghost-blog/ Blog cover image https://commons.wikimedia.org/wiki/File:Turing_Machine_Model_Davey_2012.jpg 55x250px http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ https://themes.ghost.org/docs ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] e229fb59339973f25b6f644b40f763ec73bf536c 2905 2902 2018-03-16T15:13:59Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== * Install [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node 0.8], following the standard instructions there. * Install [https://docs.ghost.org/docs/install Ghost command-line interface] * Install [https://docs.ghost.org/docs/cli-install Ghost from the command line] I created an additional user for the installation, but I'm not sure that was necessary. The standard non-root user on the server should do. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/theme$ npm install user@desktop:~/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/theme$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip ghost-theme-willow * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-html.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] dc7e5648d3a3cb633782f3f74d6f329355c5c6f9 2906 2905 2018-03-16T17:05:30Z Neil 2 /* Install MathJax and Prism */ Added examples wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== * Install [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node 0.8], following the standard instructions there. * Install [https://docs.ghost.org/docs/install Ghost command-line interface] * Install [https://docs.ghost.org/docs/cli-install Ghost from the command line] I created an additional user for the installation, but I'm not sure that was necessary. The standard non-root user on the server should do. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/theme$ npm install user@desktop:~/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/theme$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip ghost-theme-willow * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-html.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] bcd7dba641d275940cfc159a6ad5d59f9e9a0fb0 0 1650 2907 2906 2018-03-22T23:08:08Z Neil 2 /* Install a theme */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== * Install [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node 0.8], following the standard instructions there. * Install [https://docs.ghost.org/docs/install Ghost command-line interface] * Install [https://docs.ghost.org/docs/cli-install Ghost from the command line] I created an additional user for the installation, but I'm not sure that was necessary. The standard non-root user on the server should do. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-html.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] da7f1329add72d8bbaf6d76b9f31e0d50a733231 2910 2907 2018-03-30T13:00:59Z Neil 2 /* Installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== * Install [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node 0.8], following the standard instructions there. * Install [https://docs.ghost.org/docs/install Ghost command-line interface] * Install [https://docs.ghost.org/docs/cli-install Ghost from the command line], installing the blog in <code>/var/www/blog.domain.tld</code>. I created an additional user for the installation, but I'm not sure that was necessary. The standard non-root user on the server should do. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-html.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] 5b720bdf7af4d7e0769e707f76fcc7b5743e7749 2911 2910 2018-04-01T09:12:00Z Neil 2 /* Install MathJax and Prism */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== * Install [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node 0.8], following the standard instructions there. * Install [https://docs.ghost.org/docs/install Ghost command-line interface] * Install [https://docs.ghost.org/docs/cli-install Ghost from the command line], installing the blog in <code>/var/www/blog.domain.tld</code>. I created an additional user for the installation, but I'm not sure that was necessary. The standard non-root user on the server should do. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] 6afd31e1e909d1d8839354c787c0e2979ab65464 2932 2911 2018-10-31T11:14:57Z Neil 2 Added instructions for additional blogs on the same server wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== * Install [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node 0.8], following the standard instructions there. * Install [https://docs.ghost.org/docs/install Ghost command-line interface] * Install [https://docs.ghost.org/docs/cli-install Ghost from the command line], installing the blog in <code>/var/www/blog.domain.tld</code>. I created an additional user for the installation, but I'm not sure that was necessary. The standard non-root user on the server should do. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. ===Installing additional blogs on the same server=== This is much the same as with installing the first blog, but you must override the defaults for some settings. Before installing a second blog on the same server, manually create the database and the database user: user@server:~$ mysql -u root -p mysql> create database www_other_blog_org_prod; mysql> create user 'ghost_otherblog'@'localhost' identified by 'secretpassword'; grant all privileges on www_other_blog_org_prod.* to 'ghost_otherblog'@'localhost'; Then run the installation as normal: ghostinstall@server:/var/www/otherblog$ ghost install When prompted, give the database details you produced before, and ask Ghost ''not'' to create the <code>ghost</code> database user. Once you've done, take a look in <code>/var/www/otherblog/config.production.json</code> for the port the local Ghost server is running on "url": "<nowiki>https://www.otherblog.org</nowiki>", "server": { "port": 2369, "host": "127.0.0.1" }, Use that port number (<code>2369</code>) in the Apache proxy settings below. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] e8dde94d54c01ccf1c09b793689c791990916be8 0 1594 2908 2858 2018-03-29T22:23:51Z Neil 2 /* Websites */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ * Create a new user for <code>awstats</code>, make it part of the <code>www-data</code> and <code>adm</code> groups. Give it permissions for the awstats data files. root@desktop:~# useradd -g www-data -G adm awstats root@desktop:~# chown -R awstats:www-data /var/lib/awstats * Make the cron jobs run as this user. Adjust <code>/etc/cron.d/awstats</code> to change the <code>www-data</code> user to <code>awstats</code> MAILTO=root */10 * * * * awstats [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * awstats [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh == AWStats configuration == ==SiteDomain== AWStats doesn't like not knowing a default domain, even though the domain is specified in each site-specific configuration below. Modify <code>/etc/awstats/awstats.conf</code> to include a default domain, such as the one that serves the AWStats pages. SiteDomain="site.domain.tld" === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update root@desktop:~# touch /var/cache/awstats/site2.domain.tld root@desktop:~# chown awstats:www-data /var/cache/awstats/site2.domain.tld :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# systemctl reload apache2.service == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == Cataloguing old stats and logs == If you want to keep historic stats from an old machine when moving to a new one, you should be able to move the files in <code>DirData</code> in the config files, typically <code>/var/lib/awstats</code>. However, that didn't work for me, so I recreated the stats from the old log files. * Disable automatic updates while you're doing this. Edit <code>/etc/cron.d/awstats</code> and comment out the two commands there. * Merge all the logs you want into one file: root@desktop:~# perl /usr/share/awstats/tools/logresolvemerge.pl /var/log/apache2/site.access.log* > site.all.log * Mail logs need to be manually strung together in date order, converted, and bundled together. root@desktop:~# rm mail.converted.log root@desktop:~# for f in `ls -tr /var/log/mail.log*` ; do /usr/share/doc/awstats/examples/maillogconvert.pl standard < $f >> mail.converted.log; done * Process them (this will take a long time): root@desktop:~# /var/www/cgi-bin.scripts.domain.tld/awstats.pl -update -config=domain.tld -LogFile=site.all.log == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] 58e6d1b6332745a3960c19b246899703b8743d7d 2909 2908 2018-03-30T07:26:39Z Neil 2 /* Websites */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Samba setup|Samba]] |align="center" width="50%"| [[Backup setup|Backup]] |} I use [http://awstats.sourceforge.net/ AWStats] to keep track of the usage of my web sites and mail server. Setting it up is quite simple == Installation == * Install AWStats root@desktop:~# apt-get install awstats libnet-ip-perl libgeo-ipfree-perl * Because I have the <code>cgi-bin</code> directory in a non-standard location, copy the awstats script to that location: root@desktop:~# cp /usr/lib/cgi-bin/awstats.pl /var/www/cgi.site.domain.tld/ * Create a new user for <code>awstats</code>, make it part of the <code>www-data</code> and <code>adm</code> groups. Give it permissions for the awstats data files. root@desktop:~# useradd -g www-data -G adm awstats root@desktop:~# chown -R awstats:www-data /var/lib/awstats * Make the cron jobs run as this user. Adjust <code>/etc/cron.d/awstats</code> to change the <code>www-data</code> user to <code>awstats</code> MAILTO=root */10 * * * * awstats [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * awstats [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh == AWStats configuration == ==SiteDomain== AWStats doesn't like not knowing a default domain, even though the domain is specified in each site-specific configuration below. Modify <code>/etc/awstats/awstats.conf</code> to include a default domain, such as the one that serves the AWStats pages. SiteDomain="site.domain.tld" === Websites === * Each site you want to generate a log view for needs its own configuration. For the site <code>site.domain.tld</code>. copy the basic config file <code>/etc/awstats/awstats.conf</code> to <code>/etc/awstats/awstats.site.domain.tld.conf</code>. Make the following changes LogFile="/var/log/apache2/site.domain.tld.access.log" SiteDomain="site.domain.tld" HostAliases="localhost 127.0.0.1 site.domain.tld" LogFormat=1 LoadPlugin="tooltips" LoadPlugin="geoipfree" * Create the statistics databases root@desktop:~# /var/www/cgi.site.domain.tld/awstats.pl -config=site2.domain.tld -update root@desktop:~# mkdir /var/cache/awstats/site2.domain.tld root@desktop:~# chown awstats:www-data /var/cache/awstats/site2.domain.tld :Repeat for each site you have * The default cron job for AwStats, <code>/etc/cron.d/awstats</code>,works: it updates the stats every ten minutes. MAILTO=root */10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh # Generate static reports: 10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh :Note that the cron jobs need to run as <code>root</code>, not <code>www-data</code>. === Mail servers === This is very similar to the setup for Web log statistics, but the config file is a bit different. This assumes you're using [[Postfix server setup|Postfix]] as the mail server. * Create the config file <code>/etc/awstats/awstats.mail.conf</code> with the following changes: # MAIN SETUP SECTION (Required to make AWStats work) LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |" LogType=M LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd" SiteDomain="mail.domain.tld" HostAliases="localhost 127.0.0.1 mail.domain.tld smtp.domain.tld" LoadPlugin="tooltips" LoadPlugin="geoipfree" # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) LevelForBrowsersDetection=0 LevelForOSDetection=0 LevelForRefererAnalyze=0 LevelForRobotsDetection=0 LevelForWormsDetection=0 LevelForSearchEnginesDetection=0 LevelForFileTypesDetection=0 # OPTIONAL ACCURACY SETUP SECTION (Not required but increase AWStats features) ShowMenu=1 ShowSummary=HB ShowMonthStats=HB ShowDaysOfMonthStats=HB ShowDaysOfWeekStats=HB ShowHoursStats=HB ShowDomainsStats=0 ShowHostsStats=HBL ShowAuthenticatedUsers=0 ShowRobotsStats=0 ShowEMailSenders=HBML ShowEMailReceivers=HBML ShowSessionsStats=0 ShowPagesStats=0 ShowFileTypesStats=0 ShowFileSizesStats=0 ShowBrowsersStats=0 ShowOSStats=0 ShowOriginStats=0 ShowKeyphrasesStats=0 ShowKeywordsStats=0 ShowMiscStats=0 ShowHTTPErrorsStats=0 ShowSMTPErrorsStats=1 ===Remove default config=== If the default <code>awstats.conf</code> remains in <code>/etc/awstats</code>, it will generate errors as <code>SiteDomain</code> isn't set. Therefore, rename it. root@desktop:~# mv /etc/awstats/awstats.conf /etc/awstats/awstats.conf.original == Apache configuration == * Allow whatever virtual host you're using to view the statistics. Update <code>site.domain.tld</code> to include these lines inside the <code>VirtualHost</code> block Alias /awstatsclasses "/usr/share/awstats/lib/" Alias /awstats-icon/ "/usr/share/awstats/icon/" Alias /awstatscss "/usr/share/doc/awstats/examples/css" ScriptAlias /statistics/ /var/www/cgi-bin.domain.tld/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatch <Files "awstats.pl"> AuthType Basic AuthName "AWStats" require valid-user AuthUserFile /etc/apache2/awstats.passwd </Files> * The final stanza places the statistics behind a password (though only one password is needed to view the statistics of any of the sites). Add passwords for viewing the statistics with hte command root@desktop:~# htpasswd -c /etc/apache2/awstats.passwd user_name :where user_name is the relevant user and then give the password. * Reload Apache config root@desktop:~# systemctl reload apache2.service == View stats == You can now view the statistics by browsing to <code><nowiki>http://site.domain.tld/statistics/awstats.pl?config=site2.domain.tld</nowiki></code>, where <code>site2.domain.tld</code> is the name of one of the AWStats config files created earlier. == Cataloguing old stats and logs == If you want to keep historic stats from an old machine when moving to a new one, you should be able to move the files in <code>DirData</code> in the config files, typically <code>/var/lib/awstats</code>. However, that didn't work for me, so I recreated the stats from the old log files. * Disable automatic updates while you're doing this. Edit <code>/etc/cron.d/awstats</code> and comment out the two commands there. * Merge all the logs you want into one file: root@desktop:~# perl /usr/share/awstats/tools/logresolvemerge.pl /var/log/apache2/site.access.log* > site.all.log * Mail logs need to be manually strung together in date order, converted, and bundled together. root@desktop:~# rm mail.converted.log root@desktop:~# for f in `ls -tr /var/log/mail.log*` ; do /usr/share/doc/awstats/examples/maillogconvert.pl standard < $f >> mail.converted.log; done * Process them (this will take a long time): root@desktop:~# /var/www/cgi-bin.scripts.domain.tld/awstats.pl -update -config=domain.tld -LogFile=site.all.log == See also == * [http://maestric.com/doc/ubuntu/awstats Install Awstats on Ubuntu] * [http://awstats.sourceforge.net/#DOC AWStats documentation] * [http://awstats.sourceforge.net/docs/awstats_faq.htm AWStats FAQ] [[Category:Server setup]] c1e801caa0d9c257ba3f69b8965bb29d3a82409e 0 1531 2912 2894 2018-07-16T08:27:24Z Neil 2 /* Basic installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL.) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 9832313c9a1ad2407a5734836d5d8b17d93a0dd3 2930 2912 2018-10-18T08:03:58Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc.pid #Debian default Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] d5be7f4b10fddfcbc4c60ec000f04b4b263fdfb3 2953 2930 2020-08-26T11:40:54Z Neil 2 /* DMARC */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include the single <code>SOCKET</code> line at the end. # Command-line options specified here will override the contents of # /etc/opendkim.conf. See opendkim(8) for a complete list of options. #DAEMON_OPTS="" # # Uncomment to specify an alternate socket # Note that setting this will override any Socket value in opendkim.conf #SOCKET="local:/var/run/opendkim/opendkim.sock" # default #SOCKET="inet:54321" # listen on all interfaces on port 54321 #SOCKET="inet:12345@localhost" # listen on loopback on port 12345 #SOCKET="inet:12345@192.0.2.1" # listen on 192.0.2.1 on port 12345 # listen on loopback on port 8891 SOCKET="inet:8891@localhost" :(note that <code>systemctl</code> doesn't like comments on the ends of command lines.) * Create the directories for the keys (one for each domain) mkdir /etc/opendkim mkdir /etc/opendkim/example.com mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] df5cd663bb6da40dda909802a6dac58ca62a70ac 2955 2953 2020-11-09T17:52:11Z Neil 2 /* DKIM */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 1749e7a9359a7e522ddb8810c3fb5e02295b55e8 2956 2955 2020-11-12T11:40:41Z Neil 2 /* Amavis + ClamAV + SpamAssassin */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/20-debian-defaults</code> and <code>/etc/amavis/conf.d/21-ubuntu-defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] f1a397bad096c36ccacbabbbb46dafe5403f0bba 0 1526 2913 2904 2018-08-20T10:02:08Z Neil 2 /* Change tooltip colours */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) [[Category:Server setup]] 9a7d42a6351c572a4e22f5acbe008f58e25598a8 2915 2913 2018-08-20T10:17:33Z Neil 2 /* Install extra system monitoring tools */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) [[Category:Server setup]] 50ac3f88e4186c1a9d072eab12f570323b99ba3a 2916 2915 2018-09-01T10:37:50Z Neil 2 /* Media server */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://spideroak.com/opendownload/ Spideroak] (and the [https://spideroak.com/blog/20130920145427-ubuntudebian-apt-repository-gpg-key-update repo key]) * [http://www.rstudio.com/ide/download/desktop RStudio] VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) ==fzf for command line fuzzy search== A better Bash history finder (and general file finder) than the standard search. Install as a local user using the command line. Instructions at the [https://github.com/junegunn/fzf fzf page] [[Category:Server setup]] c08335f66b72991e762dd79ff1537b7101249a9f 2917 2916 2018-09-01T10:38:42Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [http://www.rstudio.com/ide/download/desktop RStudio] * [https://github.com/ggreer/the_silver_searcher fzf] for fuzzy search VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) ==fzf for command line fuzzy search== A better Bash history finder (and general file finder) than the standard search. Install as a local user using the command line. Instructions at the [https://github.com/junegunn/fzf fzf page] [[Category:Server setup]] 06eed89d35ad146af1fa6959709dd2e21a4f8443 2918 2917 2018-09-01T10:39:09Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [http://www.rstudio.com/ide/download/desktop RStudio] * [https://github.com/ggreer/the_silver_searcher fzf] for fuzzy search VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) [[Category:Server setup]] 1727f86d294cb0bbc0c8a796ca94ab7c663955a1 2919 2918 2018-09-01T10:56:21Z Neil 2 /* Normal packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) (Generate the package list with user@desktop:~$ comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) > manually-installed-packages.txt ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [http://www.rstudio.com/ide/download/desktop RStudio] * [https://github.com/ggreer/the_silver_searcher fzf] for fuzzy search VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) [[Category:Server setup]] c11a5af7943c4e9adbd8acbd1cd45b83f6473196 2943 2919 2020-02-14T13:18:06Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) (Generate the package list with user@desktop:~$ comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) > manually-installed-packages.txt ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * Google Earth * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://ccl.northwestern.edu/netlogo/download.shtml NetLogo 3.1.5] (registration required) * [http://www.ra.cs.uni-tuebingen.de/software/JavaNNS/ JavaNNS] * [http://sbmase.sourceforge.net/ SB-MASE] * [https://launchpad.net/~nvbn-rm/+archive/ppa Everpad] from the PPA * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [http://www.rstudio.com/ide/download/desktop RStudio] * [https://github.com/junegunn/fzf fzf] for fuzzy search VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) [[Category:Server setup]] c9280eca18e8feb3c0e008def7d9ab92606acb74 2945 2943 2020-02-14T17:02:05Z Neil 2 /* Downloads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) (Generate the package list with user@desktop:~$ comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) > manually-installed-packages.txt ) Note that many of them may already be installed from previous steps. === Downloads === Install these from normal downloads: * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://github.com/junegunn/fzf fzf] for fuzzy search * [https://www.sublimetext.com/docs/3/linux_repositories.html Sublime Text and Sublime Merge] * [https://snapcraft.io/slack Slack] as a Snap * [https://tresorit.com/download/linux Tresorit] for secure cloud backups VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) [[Category:Server setup]] adbcec20ff1675392aa87fe7d5f1f30d223dbe05 2949 2945 2020-04-29T11:53:31Z Neil 2 /* Install additional packages */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) (Generate the package list with user@desktop:~$ comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) > manually-installed-packages.txt ) Note that many of them may already be installed from previous steps. === Snap packages === The command user@desktop:~$ snap list will list all the installed snap packages. So far, I have <code>discord</code>, <code>postman</code> and <code>slack</code> installed. === Downloads === Install these from normal downloads: * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://github.com/junegunn/fzf fzf] for fuzzy search * [https://www.sublimetext.com/docs/3/linux_repositories.html Sublime Text and Sublime Merge] * [https://snapcraft.io/slack Slack] as a Snap * [https://tresorit.com/download/linux Tresorit] for secure cloud backups VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server --with-library ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) [[Category:Server setup]] fbd1c263afc61e37743605abd60f416f2bec1dbd 0 1543 2914 2863 2018-08-20T10:03:26Z Neil 2 /* Set up ufw */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} I used to do [[#Server (Old)|firewall setup manually]] for the server, but <code>ufw</code> has now improved to the point where I use that. See historic versions of this page from before December 2016 for details. ==Set up ufw== Enable ufw and add rules for the services the server machine will provide: root@server:~# ufw enable root@server:~# ufw allow ssh root@server:~# ufw allow http root@server:~# ufw allow https root@server:~# ufw allow domain root@server:~# ufw allow ipp root@server:~# ufw allow imap root@server:~# ufw allow smtp root@server:~# ufw allow submission root@server:~# ufw allow from 192.168.1.0/24 to any port mysql root@server:~# ufw allow from 192.168.1.0/24 to any port postgresql root@server:~# ufw allow git root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 27017:27019 # Mongo root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 28017 # Mongo web interface Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto udp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 3551 (The last rules are for [https://community.kde.org/KDEConnect KDE Connect], [https://github.com/MatejVancik/amaroKontrol amaroKontrol], and [https://help.ubuntu.com/community/apcupsd APC UPS] respectively). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 1714:1764/tcp ALLOW IN 192.168.1.0/24 8484/tcp ALLOW IN 192.168.1.0/24 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) [[Category:Server setup]] fb79ce681a7b2e2071534a36614c9ab94b40c027 2948 2914 2020-03-19T22:37:22Z Neil 2 /* Set up ufw */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |align="center" width="50%"| [[Protect SSH]] |} I used to do [[#Server (Old)|firewall setup manually]] for the server, but <code>ufw</code> has now improved to the point where I use that. See historic versions of this page from before December 2016 for details. ==Set up ufw== Enable ufw and add rules for the services the server machine will provide: root@server:~# ufw enable root@server:~# ufw allow ssh root@server:~# ufw allow http root@server:~# ufw allow https root@server:~# ufw allow domain root@server:~# ufw allow ipp root@server:~# ufw allow imap root@server:~# ufw allow smtp root@server:~# ufw allow submission root@server:~# ufw allow from 192.168.1.0/24 to any port mysql root@server:~# ufw allow from 192.168.1.0/24 to any port postgresql root@server:~# ufw allow git root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 27017:27019 # Mongo root@server:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 28017 # Mongo web interface Enable ufw and add rules for the services the desktop machine will provide: root@desktop:~# ufw enable root@desktop:~# ufw allow ssh root@desktop:~# ufw allow http root@desktop:~# ufw allow https root@desktop:~# ufw allow domain root@desktop:~# ufw allow ipp root@desktop:~# ufw allow netbios-ns root@desktop:~# ufw allow netbios-dgm root@desktop:~# ufw allow netbios-ssn root@desktop:~# ufw allow microsoft-ds root@desktop:~# ufw allow in proto udp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 1714:1764 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8484 root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 3551 root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 (The last rules are for [https://community.kde.org/KDEConnect KDE Connect], [https://github.com/MatejVancik/amaroKontrol amaroKontrol], [https://help.ubuntu.com/community/apcupsd APC UPS], and [[Miscellaneous setup#Enable Calibre server|Calibre ebook server]] respectively). ===Check ufw's status=== root@desktop:~# ufw status verbose Status: active Logging: on (low) Default: deny (incoming), allow (outgoing), disabled (routed) New profiles: skip To Action From -- ------ ---- 22 ALLOW IN Anywhere 80 ALLOW IN Anywhere 443 ALLOW IN Anywhere 53 ALLOW IN Anywhere 631 ALLOW IN Anywhere 137 ALLOW IN Anywhere 138 ALLOW IN Anywhere 139 ALLOW IN Anywhere 445 ALLOW IN Anywhere 1714:1764/tcp ALLOW IN 192.168.1.0/24 8484/tcp ALLOW IN 192.168.1.0/24 22 (v6) ALLOW IN Anywhere (v6) 80 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6) 53 (v6) ALLOW IN Anywhere (v6) 631 (v6) ALLOW IN Anywhere (v6) 137 (v6) ALLOW IN Anywhere (v6) 138 (v6) ALLOW IN Anywhere (v6) 139 (v6) ALLOW IN Anywhere (v6) 445 (v6) ALLOW IN Anywhere (v6) [[Category:Server setup]] 98c1ba4c07775e2279ca67e41763164beb33b7eb 0 1631 2920 2819 2018-10-15T13:57:42Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client On the desktop, install <code>pgadmin3</code> as well: root@desktop:~# aptitude install postgresql postgresql-client pgadmin3 == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf</code> and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/9.1/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == Updating == It's perfectly possible to have several instances of PostreSQL running at different versions, such as 9.5 and 10.1. To migrate the data and remove the old version, [https://stackoverflow.com/questions/46687645/upgrade-postgresql-from-9-6-to-10-0-on-ubuntu-16-10 follow these steps]: * Use `pg_lsclusters` to see what's running. root@desktop:~# pg_lsclusters Ver Cluster Port Status Owner Data directory Log file 9.5 main 5432 online postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log 10 main 5433 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log * Check what's in what cluster root@server::# su - postgres postgres@server:~$ psql --cluster 9.5/main postgres=# select datname from pg_database; datname ----------- template1 template0 postgres ... ... (5 rows) postgres=# \q :(where `...` indicates some more databases in the cluster) postgres@server:~$ psql --cluster 10/main psql (10.5 (Ubuntu 10.5-0ubuntu0.18.04)) postgres=# select datname from pg_database; datname ----------- postgres template1 template0 (3 rows) : this indicates that there's nothing currently in the version 10 cluster. * Stop Posgres root@server:~# systemctl stop postgresql * Destroy the existing empty cluster in the version 10 cluster root@server:~# pg_dropcluster --stop 10 main * Move the existing data to version 10 root@server:~# pg_upgradecluster -m upgrade 9.5 main * Drop the 9.5 cluster root@server:~# pg_dropcluster --stop 9.5 main * Use `aptitude` to purge all the postgresql-9.5 packages. * Restart Postgres root@server:~# systemctl start postgresql == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] 2b26f13dd348d119507823d6fd9d6efe260db64c 6 1652 2922 2018-10-15T15:30:49Z Neil 2 wikitext text/x-wiki da39a3ee5e6b4b0d3255bfef95601890afd80709 6 1653 2923 2018-10-16T14:26:17Z Neil 2 wikitext text/x-wiki da39a3ee5e6b4b0d3255bfef95601890afd80709 6 1654 2924 2018-10-16T14:30:45Z Neil 2 wikitext text/x-wiki da39a3ee5e6b4b0d3255bfef95601890afd80709 0 1534 2925 2873 2018-10-16T15:32:46Z Neil 2 /* Creating the farm */ Updated for MW 1.31 bugfix reading LocalSettings wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so you need to modify the file <code>/usr/share/mediawiki/includes/WebStart.php</code> to load the <code>LocalSettings.php</code> from the wiki's directory rather than the central installation directory. :Therefore, around line 61 in <code>/usr/share/mediawiki/includes/WebStart.php</code>, replace the line define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); with this block: $possibleCurrentDirectoryLocalSettings = getcwd().'/LocalSettings.php'; //an extra check for LocalSettings.php under current Directory incase of symlink used. if ( !defined( 'MW_CONFIG_CALLBACK' ) ) { if ( !defined( 'MW_CONFIG_FILE' ) ) { if ( is_readable($possibleCurrentDirectoryLocalSettings) ) { define( 'MW_CONFIG_FILE', $possibleCurrentDirectoryLocalSettings ); } else { define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); } } :(This fix taken from the [https://phabricator.wikimedia.org/T203061 phabricator discussion].) * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the <code>$wgEnableUploads</code> setting: $wgEnableUploads = true; === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 6347f4d08307e4603f1dab97793a49cf9043ab25 2926 2925 2018-10-16T15:35:20Z Neil 2 /* Enabling uploads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so you need to modify the file <code>/usr/share/mediawiki/includes/WebStart.php</code> to load the <code>LocalSettings.php</code> from the wiki's directory rather than the central installation directory. :Therefore, around line 61 in <code>/usr/share/mediawiki/includes/WebStart.php</code>, replace the line define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); with this block: $possibleCurrentDirectoryLocalSettings = getcwd().'/LocalSettings.php'; //an extra check for LocalSettings.php under current Directory incase of symlink used. if ( !defined( 'MW_CONFIG_CALLBACK' ) ) { if ( !defined( 'MW_CONFIG_FILE' ) ) { if ( is_readable($possibleCurrentDirectoryLocalSettings) ) { define( 'MW_CONFIG_FILE', $possibleCurrentDirectoryLocalSettings ); } else { define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); } } :(This fix taken from the [https://phabricator.wikimedia.org/T203061 phabricator discussion].) * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the settings around <code>$wgEnableUploads</code> to be: ## To enable image uploads, make sure the 'images' directory ## is writable, then set this to true: $wgEnableUploads = true; $wgUseImageMagick = true; $wgImageMagickConvertCommand = "/usr/bin/convert"; $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); $wgUploadDirectory = "/var/www/domain.tld/mediawiki/images"; $wgUploadPath = "{$wgScriptPath}/images"; ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; $wgGenerateThumbnailOnParse = true; Yes, again, there need to be explicit absolute paths on the server's filesystem. === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 79b0d9cafff444b5a882f80bfd0b1f050160f0cd 2927 2926 2018-10-16T15:35:43Z Neil 2 /* Enabling uploads */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so you need to modify the file <code>/usr/share/mediawiki/includes/WebStart.php</code> to load the <code>LocalSettings.php</code> from the wiki's directory rather than the central installation directory. :Therefore, around line 61 in <code>/usr/share/mediawiki/includes/WebStart.php</code>, replace the line define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); with this block: $possibleCurrentDirectoryLocalSettings = getcwd().'/LocalSettings.php'; //an extra check for LocalSettings.php under current Directory incase of symlink used. if ( !defined( 'MW_CONFIG_CALLBACK' ) ) { if ( !defined( 'MW_CONFIG_FILE' ) ) { if ( is_readable($possibleCurrentDirectoryLocalSettings) ) { define( 'MW_CONFIG_FILE', $possibleCurrentDirectoryLocalSettings ); } else { define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); } } :(This fix taken from the [https://phabricator.wikimedia.org/T203061 phabricator discussion].) * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ /index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the settings around <code>$wgEnableUploads</code> to be: ## To enable image uploads, make sure the 'images' directory ## is writable, then set this to true: $wgEnableUploads = true; $wgUseImageMagick = true; $wgImageMagickConvertCommand = "/usr/bin/convert"; $wgUploadDirectory = "/var/www/domain.tld/mediawiki/images"; $wgUploadPath = "{$wgScriptPath}/images"; ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; $wgGenerateThumbnailOnParse = true; Yes, again, there need to be explicit absolute paths on the server's filesystem. === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 2ff7726ad7cea461da5a2d9f18ca2c2b32a6eb4f 2928 2927 2018-10-16T15:39:21Z Neil 2 /* Rewriting paths */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so you need to modify the file <code>/usr/share/mediawiki/includes/WebStart.php</code> to load the <code>LocalSettings.php</code> from the wiki's directory rather than the central installation directory. :Therefore, around line 61 in <code>/usr/share/mediawiki/includes/WebStart.php</code>, replace the line define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); with this block: $possibleCurrentDirectoryLocalSettings = getcwd().'/LocalSettings.php'; //an extra check for LocalSettings.php under current Directory incase of symlink used. if ( !defined( 'MW_CONFIG_CALLBACK' ) ) { if ( !defined( 'MW_CONFIG_FILE' ) ) { if ( is_readable($possibleCurrentDirectoryLocalSettings) ) { define( 'MW_CONFIG_FILE', $possibleCurrentDirectoryLocalSettings ); } else { define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); } } :(This fix taken from the [https://phabricator.wikimedia.org/T203061 phabricator discussion].) * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/sitemap/ RewriteCond %{REQUEST_URI} !^/.well-known/ RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ %{DOCUMENT_ROOT}/mediawiki/index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the settings around <code>$wgEnableUploads</code> to be: ## To enable image uploads, make sure the 'images' directory ## is writable, then set this to true: $wgEnableUploads = true; $wgUseImageMagick = true; $wgImageMagickConvertCommand = "/usr/bin/convert"; $wgUploadDirectory = "/var/www/domain.tld/mediawiki/images"; $wgUploadPath = "{$wgScriptPath}/images"; ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; $wgGenerateThumbnailOnParse = true; Yes, again, there need to be explicit absolute paths on the server's filesystem. === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == Rewriting for Mediawiki 1.7 == A historical note, for when the bug in MW 1.11 gets fixed and I forget what worked before the bug! In <tt>/etc/apache2/sites-enabled/site.domain.tld</tt>: RewriteEngine on # Don't rewrite requests for files that really exist or should return 404. RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki/article</nowiki> -- this is the main rule RewriteRule ^/(.*)$ /mediawiki/index.php/$1 [L,QSA] In <tt>LocalSettings.php</tt> $wgArticlePath = "/$1"; $wgUsePathInfo = true; (the last one I'm not sure about, as it wasn't set in MediaWiki 1.7) == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 8d9b81a4f36be88c818f9c51094981cb7ab816e4 2929 2928 2018-10-16T15:39:51Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so you need to modify the file <code>/usr/share/mediawiki/includes/WebStart.php</code> to load the <code>LocalSettings.php</code> from the wiki's directory rather than the central installation directory. :Therefore, around line 61 in <code>/usr/share/mediawiki/includes/WebStart.php</code>, replace the line define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); with this block: $possibleCurrentDirectoryLocalSettings = getcwd().'/LocalSettings.php'; //an extra check for LocalSettings.php under current Directory incase of symlink used. if ( !defined( 'MW_CONFIG_CALLBACK' ) ) { if ( !defined( 'MW_CONFIG_FILE' ) ) { if ( is_readable($possibleCurrentDirectoryLocalSettings) ) { define( 'MW_CONFIG_FILE', $possibleCurrentDirectoryLocalSettings ); } else { define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); } } :(This fix taken from the [https://phabricator.wikimedia.org/T203061 phabricator discussion].) * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/sitemap/ RewriteCond %{REQUEST_URI} !^/.well-known/ RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ %{DOCUMENT_ROOT}/mediawiki/index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the settings around <code>$wgEnableUploads</code> to be: ## To enable image uploads, make sure the 'images' directory ## is writable, then set this to true: $wgEnableUploads = true; $wgUseImageMagick = true; $wgImageMagickConvertCommand = "/usr/bin/convert"; $wgUploadDirectory = "/var/www/domain.tld/mediawiki/images"; $wgUploadPath = "{$wgScriptPath}/images"; ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; $wgGenerateThumbnailOnParse = true; Yes, again, there need to be explicit absolute paths on the server's filesystem. === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Set MathML as default rendering option $wgDefaultUserOptions['math'] = 'mathml'; $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 82cab40cb23384824e113ea89b15e296fc0cf306 2931 2929 2018-10-27T16:34:18Z Neil 2 /* Enabling equation display */ Updated for MW 1.31, using texvc wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so you need to modify the file <code>/usr/share/mediawiki/includes/WebStart.php</code> to load the <code>LocalSettings.php</code> from the wiki's directory rather than the central installation directory. :Therefore, around line 61 in <code>/usr/share/mediawiki/includes/WebStart.php</code>, replace the line define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); with this block: $possibleCurrentDirectoryLocalSettings = getcwd().'/LocalSettings.php'; //an extra check for LocalSettings.php under current Directory incase of symlink used. if ( !defined( 'MW_CONFIG_CALLBACK' ) ) { if ( !defined( 'MW_CONFIG_FILE' ) ) { if ( is_readable($possibleCurrentDirectoryLocalSettings) ) { define( 'MW_CONFIG_FILE', $possibleCurrentDirectoryLocalSettings ); } else { define( 'MW_CONFIG_FILE', "$IP/LocalSettings.php" ); } } :(This fix taken from the [https://phabricator.wikimedia.org/T203061 phabricator discussion].) * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/sitemap/ RewriteCond %{REQUEST_URI} !^/.well-known/ RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ %{DOCUMENT_ROOT}/mediawiki/index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the settings around <code>$wgEnableUploads</code> to be: ## To enable image uploads, make sure the 'images' directory ## is writable, then set this to true: $wgEnableUploads = true; $wgUseImageMagick = true; $wgImageMagickConvertCommand = "/usr/bin/convert"; $wgUploadDirectory = "/var/www/domain.tld/mediawiki/images"; $wgUploadPath = "{$wgScriptPath}/images"; ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; $wgGenerateThumbnailOnParse = true; Yes, again, there need to be explicit absolute paths on the server's filesystem. === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Enable the various rendering modes # $wgMathValidModes[] = 'mathml'; $wgMathValidModes[] = 'latex'; // Set the default rendering option # $wgDefaultUserOptions['math'] = 'mathml'; # $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; wgDefaultUserOptions['math'] = 'latex'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] f5af7385debd7acf607691b5b7caa75d9df6429a 2933 2931 2018-11-02T14:52:59Z Neil 2 /* Creating the farm */ Updated for MW 1.31 bugfix reading LocalSettings wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki SetEnv MW_INSTALL_PATH /var/www/site.domain.tld/mediawiki # Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so need to set the MW_INSTALL_PATH explicitly <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(The setting of <code>MW_INSTALL_PATH</code> is taken from the [https://phabricator.wikimedia.org/T203061#4551077 phabricator discussion].) * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/sitemap/ RewriteCond %{REQUEST_URI} !^/.well-known/ RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ %{DOCUMENT_ROOT}/mediawiki/index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the settings around <code>$wgEnableUploads</code> to be: ## To enable image uploads, make sure the 'images' directory ## is writable, then set this to true: $wgEnableUploads = true; $wgUseImageMagick = true; $wgImageMagickConvertCommand = "/usr/bin/convert"; $wgUploadDirectory = "/var/www/domain.tld/mediawiki/images"; $wgUploadPath = "{$wgScriptPath}/images"; ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; $wgGenerateThumbnailOnParse = true; Yes, again, there need to be explicit absolute paths on the server's filesystem. === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Enable the various rendering modes # $wgMathValidModes[] = 'mathml'; $wgMathValidModes[] = 'latex'; // Set the default rendering option # $wgDefaultUserOptions['math'] = 'mathml'; # $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; wgDefaultUserOptions['math'] = 'latex'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 184042b91670cb5703019d873114c8ef5fff87ac 0 1535 2934 2861 2018-11-13T16:39:49Z Neil 2 /* Installing plugins */ Added composer update command wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I'm using [https://roundcube.net/ Roundcube] as a webmail interface. * Install the packages: root@server:~# aptitude install roundcube roundcube-pgsql roundcube-plugins :(Install other <code>aspell-</code> dictionaries as you wish at the same time.) * Follow the defaults in the questions asked during installation. However, these questions don't address all the configuration options. Therefore, after installation, reconfigure the packages: root@server:~# sudo dpkg-reconfigure roundcube-core * Roundcube wants to make itself available as <code>domain.tld/roundcube</code> on all sites. The file <code>/etc/apache2/conf-available/roundcube.conf</code> does this with the <code>Alias /roundcube /var/lib/roundcube</code> directive, which is initially commented out. Keep it commented out, and copy the contents of that file into the <code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>, uncommenting the <code>Alias</code> line. * Disable the Roundcube configuration, enable the webmail site, and reload Apache: root@server:~# a2disconf roundcube root@server:~# a2ensite webmail.domain.tld.conf root@server:~# systemctl reload apache2.service * Visit <code><nowiki>https://webmail.domain.tld</nowiki></code> and you should be able to log on. ==Installing plugins== Where possible, use the [https://plugins.roundcube.net/ Roundcube plugins installer], working from the <code>/var/lib/roundcube</code> directory. ===Mobile skin=== Roundcube doesn't work well on mobile devices. The [https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile Melanie2 skin] seems to be a good alternative. The [https://roundcubeinbox.wordpress.com/2016/04/26/roundcube-for-mobile-devices/ instructions at the Roundcube inbox] are slightly unclear with paths, so here's what I did. root@server:~# cd /var/lib/roundcube root@server:/var/lib/roundcube# git clone https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile.git skins/melanie2_larry_mobile root@server:/var/lib/roundcube# php composer.phar update root@server:/var/lib/roundcube# php composer.phar require melanie2/mobile:dev-master :remembering to say "no" to activating the <code>jquery_mobile</code> plugin, but "yes" to activating the <code>mobile</code> plugin. ===Enigma=== The Enigma plugin comes as part of the Roundcube installation. To enable it, add <code>enigma</code> to the list of enabled plugins in <code>/etc/roundcube/config.inc.php</code>. (Note that this doesn't seem to work at the moment.) ==<code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>== <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On # SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/fullchain.pem DocumentRoot /var/lib/roundcube ServerName webmail.domain.tld <Directory /> Options +FollowSymLinks -Indexes AllowOverride None </Directory> <Directory /var/lib/roundcube/> Options +FollowSymLinks # This is needed to parse /var/lib/roundcube/.htaccess. See its # content before setting AllowOverride to None. AllowOverride All <IfVersion >= 2.3> Require all granted </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all </IfVersion> </Directory> # Protecting basic directories: <Directory /var/lib/roundcube/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/lib/roundcube/temp> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> <Directory /var/lib/roundcube/logs> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/webmail.domain.tld.access.log combined ServerSignature Off </VirtualHost> == See also == * [https://github.com/roundcube/roundcubemail/wiki Roundcube documentation] * [https://help.ubuntu.com/community/Roundcube Installing Roundcube on Ubuntu] [[Category:Server setup]] dcce5e6928122c9c3af2a1853a0cbef7b3150fdb 2935 2934 2018-11-13T16:43:26Z Neil 2 /* Installing plugins */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I'm using [https://roundcube.net/ Roundcube] as a webmail interface. * Install the packages: root@server:~# aptitude install roundcube roundcube-pgsql roundcube-plugins :(Install other <code>aspell-</code> dictionaries as you wish at the same time.) * Follow the defaults in the questions asked during installation. However, these questions don't address all the configuration options. Therefore, after installation, reconfigure the packages: root@server:~# sudo dpkg-reconfigure roundcube-core * Roundcube wants to make itself available as <code>domain.tld/roundcube</code> on all sites. The file <code>/etc/apache2/conf-available/roundcube.conf</code> does this with the <code>Alias /roundcube /var/lib/roundcube</code> directive, which is initially commented out. Keep it commented out, and copy the contents of that file into the <code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>, uncommenting the <code>Alias</code> line. * Disable the Roundcube configuration, enable the webmail site, and reload Apache: root@server:~# a2disconf roundcube root@server:~# a2ensite webmail.domain.tld.conf root@server:~# systemctl reload apache2.service * Visit <code><nowiki>https://webmail.domain.tld</nowiki></code> and you should be able to log on. ==Installing plugins== Where possible, use the [https://plugins.roundcube.net/ Roundcube plugins installer], working from the <code>/var/lib/roundcube</code> directory. ===Mobile skin=== Roundcube doesn't work well on mobile devices. The [https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile Melanie2 skin] seems to be a good alternative. The [https://roundcubeinbox.wordpress.com/2016/04/26/roundcube-for-mobile-devices/ instructions at the Roundcube inbox] are slightly unclear with paths, so here's what I did. root@server:~# cd /var/lib/roundcube root@server:/var/lib/roundcube# git clone <nowiki>https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile.git</nowiki> skins/melanie2_larry_mobile root@server:/var/lib/roundcube# php composer.phar update root@server:/var/lib/roundcube# php composer.phar require melanie2/mobile:dev-master :remembering to say "no" to activating the <code>jquery_mobile</code> plugin, but "yes" to activating the <code>mobile</code> plugin. ===Enigma=== The Enigma plugin comes as part of the Roundcube installation. To enable it, add <code>enigma</code> to the list of enabled plugins in <code>/etc/roundcube/config.inc.php</code>. (Note that this doesn't seem to work at the moment.) ==<code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>== <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On # SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/fullchain.pem DocumentRoot /var/lib/roundcube ServerName webmail.domain.tld <Directory /> Options +FollowSymLinks -Indexes AllowOverride None </Directory> <Directory /var/lib/roundcube/> Options +FollowSymLinks # This is needed to parse /var/lib/roundcube/.htaccess. See its # content before setting AllowOverride to None. AllowOverride All <IfVersion >= 2.3> Require all granted </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all </IfVersion> </Directory> # Protecting basic directories: <Directory /var/lib/roundcube/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/lib/roundcube/temp> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> <Directory /var/lib/roundcube/logs> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/webmail.domain.tld.access.log combined ServerSignature Off </VirtualHost> == See also == * [https://github.com/roundcube/roundcubemail/wiki Roundcube documentation] * [https://help.ubuntu.com/community/Roundcube Installing Roundcube on Ubuntu] [[Category:Server setup]] 150a6375dcf4f6a0aada56146417fb52ae90ca91 0 1 2936 2727 2019-01-26T12:57:34Z Neil 2 Added a couple of social media links wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == <div style="float: right;"> {| {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://wheretofind.me/@NeilNjae NeilNjae] on WhereToFindMe * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://ello.co/neilnjae @neilnjae] on Ello * [https://keybase.io/neilnjae NeilNjae] on Keybase.io * [https://pluspora.com/people/e1c57380ade5013646c5005056268def NeilNjae] on Plusporia * [https://mewe.com/profile/5bbbb775a5f4e527d1b3d9e5 NeilNjae] on MeWe * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] * Send me BitCoins at [https://blockchain.info/address/1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV 1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV] |} </div> Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://tresorit.com/ Tresorit] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 851a33af6e8d1a8e31c4e32278816090f9837538 2939 2936 2019-12-16T19:11:11Z Neil 2 Added some headings wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == <div style="float: right;"> {| {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://wheretofind.me/@NeilNjae NeilNjae] on WhereToFindMe * [https://plus.google.com/+NeilSmith Neil Smith] on Google+ * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://ello.co/neilnjae @neilnjae] on Ello * [https://keybase.io/neilnjae NeilNjae] on Keybase.io * [https://pluspora.com/people/e1c57380ade5013646c5005056268def NeilNjae] on Plusporia * [https://mewe.com/profile/5bbbb775a5f4e527d1b3d9e5 NeilNjae] on MeWe * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] * Send me BitCoins at [https://blockchain.info/address/1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV 1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV] |} </div> === [[:Category:RPGs|RPGs]] === Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. === [[server setup|Servers and sysadmin]] === I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). === Other stuff === I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://tresorit.com/ Tresorit] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. d6ece5b21106e25b2c2c0ffc2984078cae2e0253 2957 2939 2020-11-12T11:48:00Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == <div style="float: right;"> {| {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://wheretofind.me/@NeilNjae NeilNjae] on WhereToFindMe * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github <!-- * [https://ello.co/neilnjae @neilnjae] on Ello --> * [https://keybase.io/neilnjae NeilNjae] on Keybase.io <!-- * [https://pluspora.com/people/e1c57380ade5013646c5005056268def NeilNjae] on Plusporia --> <!-- * [https://mewe.com/profile/5bbbb775a5f4e527d1b3d9e5 NeilNjae] on MeWe --> * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] * Send me BitCoins at [https://blockchain.info/address/1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV 1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV] |} </div> === [[:Category:RPGs|RPGs]] === Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. === [[server setup|Servers and sysadmin]] === I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). === Other stuff === I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://tresorit.com/ Tresorit] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 7858d3ba524b2f951a422593a795c24e5afbc5a1 0 1630 2937 2714 2019-02-26T16:37:48Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity build-essential calibre couchdb-bin create-resources curl dos2unix exfat-utils fbreader finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gnome-games gnubg gnuplot-qt # Possible conflict with gnuplot-nox gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve hal-info haskell-mode haskell-platform haskell-platform-prof llvm ibus inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs npm normalize-audio octave ogmtools openshot p7zip pandoc pavucontrol pavumeter pdftk pitivi postgresql postgresql-client r-base sbcl scratch screen scribus scribus-template seahorse shotwell slime sqlite3 stellarium swi-prolog tesseract-ocr texlive-full texlive-bibtex-extra texlive-pictures texlive-science tk-dev tk tofrodos traceroute units vino virtuoso-server vlc webp whois wireshark xine-plugin xine-ui xinetd xtightvncviewer zenity Packages which were either already installed by default, or no longer necessary. # biblatex # blueman # cheese # cmake # cython # dpkg-dev # easytag ? # eog # ffmpeg # fgetty # fop # gmusicbrowser # gucharmap # guvcview # html2text # idle # idle3 # paprefs # pastebinit # perlmagick # python-all # python3-all # r-recommended # samba # sni-qt # software-center # software-properties-gtk # sound-juicer # thunderbird # tuxpaint # tuxpaint-config # xsane 2ff183d36f02889241a9c092eb821e394929e880 2941 2937 2020-02-10T14:46:52Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] acidrip arduino arduino-mk at audacity build-essential calibre couchdb-bin create-resources curl dos2unix exfat-utils fbreader finger freemind freemind-plugins-help freemind-plugins-script freemind-plugins-svg fritzing fzf gconf-editor geogebra ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gmtp gnome-games gnubg gnuplot-qt # Possible conflict with gnuplot-nox gocr gocr-tk gparted graphicsmagick gtk2-engines-qtcurve hal-info haskell-mode haskell-platform haskell-platform-prof llvm ibus inkscape jabref kdegames-card-data-extra kdiff3 kid3 kile kile-doc kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui mtp-tools mtpfs mysql-client mysql-server nmap nodejs npm normalize-audio octave ogmtools openshot p7zip pandoc pavucontrol pavumeter pdftk pitivi postgresql postgresql-client r-base sbcl scratch screen scribus scribus-template seahorse shotwell silversearcher-ag slime sqlite3 stellarium swi-prolog tesseract-ocr texlive-full texlive-bibtex-extra texlive-pictures texlive-science tk-dev tk tofrodos traceroute units vino virtuoso-server vlc webp whois wireshark xine-plugin xine-ui xinetd xtightvncviewer zenity Packages which were either already installed by default, or no longer necessary. # biblatex # blueman # cheese # cmake # cython # dpkg-dev # easytag ? # eog # ffmpeg # fgetty # fop # gmusicbrowser # gucharmap # guvcview # html2text # idle # idle3 # paprefs # pastebinit # perlmagick # python-all # python3-all # r-recommended # samba # sni-qt # software-center # software-properties-gtk # sound-juicer # thunderbird # tuxpaint # tuxpaint-config # xsane 21d4ab2fdb35c857c0c1008959dc3e3010c8d968 2944 2941 2020-02-14T17:01:28Z Neil 2 wikitext text/x-wiki A list of additional packages for Kubuntu. To be read in conjunction with [[Miscellaneous setup#Normal_packages|miscellaneous setup package installation documents]]. [[Category:Server setup]] at audacity build-essential create-resources curl dos2unix exfat-utils fbreader finger gconf-editor ghex gimp gimp-data gimp-data-extras gimp-help-en gimp-plugin-registry git git-doc git-gui gitk gnome-games gnubg gparted graphicsmagick gtk2-engines-qtcurve haskell-platform haskell-platform-prof llvm kile kile-doc ibus inkscape kdiff3 kid3 kmplayer meld mencoder mkvtoolnix mkvtoolnix-gui mmv moreutils mp3check mplayer mplayer-fonts mplayer-gui nmap p7zip pandoc pavucontrol pavumeter pdftk pitivi r-base screen silversearcher-ag sqlite3 texlive-full texlive-bibtex-extra texlive-pictures texlive-science tk-dev tk tofrodos traceroute units webp whois wireshark xine-ui xinetd 179babc54778d19c981b3bcf284eb153423a42f8 0 1541 2938 2878 2019-07-10T09:53:19Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Disable NetworkManager == Change <code>/etc/NetworkManager/NetworkManager.conf</code> so that <code>managed=true</code> and there's no <code>dns=dnsmasq</code> line active: [main] plugins=ifupdown,keyfile #dns=dnsmasq [ifupdown] managed=true == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * To give a static IP to a WiFi connection using WPA Supplicant, use this interface stanza: iface wlan0 inet static address 192.168.1.101 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 wpa-essid MyWifiID wpa-psk my-secret-key # wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf # iface default inet dhcp * Ensure that <code>/etc/resolv.conf</code> is a symbolic link to <code>/run/resolvconf/resolv.conf</code>. If it's still pointing to somewhere with <code>NetworkManager</code> in the name, delete it and recreate it: root@desktop:~# rm /etc/resolv.conf root@desktop:~# ln -s /run/resolvconf/resolv.conf /etc/resolv.conf (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] 4169c411cfdc09775b45a2abb1bd6834163d4728 2942 2938 2020-02-10T17:31:49Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Update netplan == Modify the file <code>/etc/netplan/01-network-manager-all.yaml</code> to include the stanza: network: version: 2 renderer: NetworkManager ethernets: eno1: dhcp4: no dhcp6: no addresses: [192.168.1.251/24] gateway4: 192.168.1.1 nameservers: search: [domain.tld] addresses: [192.168.1.251,192.168.1.252] Apply the changes root@desktop:~# netplan apply == Disable NetworkManager == Change <code>/etc/NetworkManager/NetworkManager.conf</code> so that <code>managed=true</code> and there's no <code>dns=dnsmasq</code> line active: [main] plugins=ifupdown,keyfile #dns=dnsmasq [ifupdown] managed=true == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * To give a static IP to a WiFi connection using WPA Supplicant, use this interface stanza: iface wlan0 inet static address 192.168.1.101 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 wpa-essid MyWifiID wpa-psk my-secret-key # wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf # iface default inet dhcp * Ensure that <code>/etc/resolv.conf</code> is a symbolic link to <code>/run/resolvconf/resolv.conf</code>. If it's still pointing to somewhere with <code>NetworkManager</code> in the name, delete it and recreate it: root@desktop:~# rm /etc/resolv.conf root@desktop:~# ln -s /run/resolvconf/resolv.conf /etc/resolv.conf (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] 25960b2192f3a4052c330b8c63ffa95da6fba6a2 0 1533 2940 2868 2020-02-01T17:08:18Z Neil 2 Updated for PHP FastCGI, Apache event MPM, and HTTP2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# systemctl reload apache2.service ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# ssystemctl reload apache2.service == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# systemctl reload apache2.service == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/cert.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# systemctl restart apache2.service Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ===Note to self=== On Apache version 2.4.8 and above, replace: SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem with: SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off ==Add modules needed== root@server:~# a2enmod headers root@server:~# a2enmod cgi root@server:~# systemctl restart apache2.service == Install PHP == I install PHP in a moment for the [[Mediawiki farm setup]], but it needs tweaking first to allow HTTP2 to be enabled. * Install PHP and the FastCGI extension root@server:~# aptitude install php php-fpm php-apcu libapache2-mod-fastcgi * Enable PHP and the FastCGI modules root@server:~# a2enmod php7.2 root@server:~# a2enmod php7.2-fpm root@server:~# a2dismod php7.0 root@server:~# systemctl restart apache2 == Enable HTTP2 == To enable HTTP2, switch PHP to use the FPM module and Apache to the ''event'' handler. root@server:~# a2enmod proxy_fcgi root@server:~# a2enconf php7.2-fpm root@server:~# a2dismod mpm_prefork root@server:~# a2enmod mpm_event root@server:~# systemctl restart apache2 Test the PHP installation by creating a file <code>/var/www/site.domain.tld/test.php</code> containing <?php phpinfo(); ?> then visit that file in a browser. It should say Server API FPM/FastCGI near the top. Then enable HTTP2 by creating the file <code>/etc/apache2/conf-available/http2.conf</code> Protocols h2 http/1.1 # vim: syntax=apache ts=4 sw=4 sts=4 sr noet Then enable the module. root@server:~# a2enconf http2 root@server:~# a2enmod http2 root@server:~# systemctl restart apache2 Test it by pointing the [https://tools.keycdn.com/http2-test KeyCDN tester] at the domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] bf0a9efb63240d195fca73d41839be1773284083 2950 2940 2020-05-22T09:43:28Z Neil 2 /* Enable HTTP2 */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# systemctl reload apache2.service ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# ssystemctl reload apache2.service == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# systemctl reload apache2.service == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/cert.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# systemctl restart apache2.service Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ===Note to self=== On Apache version 2.4.8 and above, replace: SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem with: SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off ==Add modules needed== root@server:~# a2enmod headers root@server:~# a2enmod cgi root@server:~# systemctl restart apache2.service == Install PHP == I install PHP in a moment for the [[Mediawiki farm setup]], but it needs tweaking first to allow HTTP2 to be enabled. * Install PHP and the FastCGI extension root@server:~# aptitude install php php-fpm php-apcu libapache2-mod-fastcgi * Enable PHP and the FastCGI modules root@server:~# a2enmod php7.2 root@server:~# a2enmod php7.2-fpm root@server:~# a2dismod php7.0 root@server:~# systemctl restart apache2 == Enable HTTP2 == To enable HTTP2, switch PHP to use the FPM module and Apache to the ''event'' handler. root@server:~# a2enmod proxy_fcgi root@server:~# a2enconf php7.2-fpm root@server:~# a2dismod mpm_prefork root@server:~# a2enmod mpm_event root@server:~# systemctl restart apache2 Test the PHP installation by creating a file <code>/var/www/site.domain.tld/test.php</code> containing <?php phpinfo(); ?> then visit that file in a browser. It should say Server API FPM/FastCGI near the top. Then enable HTTP2 by creating the file <code>/etc/apache2/conf-available/http2.conf</code> Protocols h2 http/1.1 # vim: syntax=apache ts=4 sw=4 sts=4 sr noet Then enable the module. root@server:~# a2enconf http2 root@server:~# a2enmod http2 root@server:~# systemctl restart apache2 Test it by pointing the [https://tools.keycdn.com/http2-test KeyCDN tester] at the domain. Once the FCGI plugin is working, allow it to use more workers by modifying <code>/etc/php/7.2/fpm/pool.d/www.conf</code> to update these lines (including commenting out <code>pm.start_servers</code>: pm.max_children = 5 ; pm.start_servers = 2 pm.min_spare_servers = 2 pm.max_spare_servers = 6 Then restart the FCGI server. root@server:~# systemctl restart php7.2-fpm.service The server should now be using all the CPU while serving many concurrent requests. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] b6c6f9c1c946edf5e172d246dbbc19b37a23b3e5 2951 2950 2020-05-22T10:19:24Z Neil 2 /* Add modules needed */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[PostgreSQL configuration|PostgreSQL config]] |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |} I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various [[Mediawiki farm setup|Mediawiki]] instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a [[Webmail setup|webmail interface]] to my [[Dovecot server setup|mail server]], running over a SSL connection. [[Mediawiki farm setup|Mediawiki]] and [[webmail setup]] are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection. == Getting Apache2 running == Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with root@server:~# apt-get install apache2-mpm-worker == Name-based virtual hosts == Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page. Enable name-based virtual hosts by modifying the end of <code>/etc/apache2/ports.conf</code> to include the two <code>NameVirtualHost</code> lines: NameVirtualHost *:80 NameVirtualHost *:443 There will be one virtual host for each site. The files for each site will reside in a separate directory under <code>/var/www</code> and each site will have a separate configuration file in <code>/etc/apache2/sites-available</code>. A typical configuration file is below: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> <Directory /var/www/> Options Indexes FollowSymLinks MultiViews AllowOverride None Order allow,deny Allow from all </Directory> ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/ <Directory "/var/www/cgi.site.domain.tld"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log LogLevel warn CustomLog /var/log/apache2/site.domain.tld.access.log combined ServerSignature Off Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128 </Directory> </VirtualHost> Repeat this file for each virtual host you want to set up. Change the <code>ServerName</code>, <code>DocumentRoot</code>, and <code>CustomLog</code> settings to reflect the site's name and location of files. Also note the non-default location of the <code>cgi-bin</code> directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by <code>root</code>, and permissions of 755. Also note that the contents of the <code>/usr/share/doc/</code> directory are only available to users on the LAN. When a site is ready, enable it with the command: root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file then reload the configuration: root@server:~# systemctl reload apache2.service ===Setting a default virtual host === When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as <code>/etc/apache2/site-available/00-default.domain.tld</code>. You'll probably also want to disable the default host that comes with Apache2. root@server:~# a2dissite default root@server:~# a2ensite 00-default.domain.tld root@server:~# ssystemctl reload apache2.service == Stopping version number leakage == By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings. In <code>/etc/apache2/apache2.conf</code>, set the following directives: ServerTokens Prod ServerSignature Off In each virtual host configuration file, set: ServerSignature Off (it's already set in the sample above) Finally, reload the configuration: root@server:~# systemctl reload apache2.service == Secure HTTP == I use the Secure HTTP server for my webmail You'll need to look at the [[Webmail setup]] page for the rest of the configuration. The first step is to enable the Apache rewriting engine and the SSL module: root@server:~# a2enmod ssl Now, create the configuration file for the <code>squirrelmail.domain.tld</code> site, <code>/etc/apache2/sites-available/squirrelmail.domain.tld</code> <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName squirrelmail.domain.tld Redirect permanent / <nowiki>https://squirrelmail.domain.tld/</nowiki> CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/cert.pem DocumentRoot /var/www/www.domain.tld ServerName squirrelmail.domain.tld <Directory /> Options FollowSymLinks AllowOverride None </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined ServerSignature Off </VirtualHost> Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443. Finally, ask Apache to listen to port 443. Add this line to <code>/etc/apache2/ports.conf</code> Listen 443 Enable the site: root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld then restart the server: root@server:~# systemctl restart apache2.service Restart Apache and you should be able to see the secure site. It should show the same content as the base site, <code><nowiki>www.domain.tld</nowiki></code>. We'll do [[Webmail setup]] later. ===Note to self=== On Apache version 2.4.8 and above, replace: SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem with: SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem ==Fix security holes== * Modify <code>/etc/apache2/conf-enabled/security.conf</code> to turn off server signatures: ServerSignature Off ==Add modules needed== root@server:~# a2enmod headers root@server:~# a2enmod cgi Update <code>/etc/apache2/mods-enabled/deflate.conf</code> to include <code>text/javascript</code> in the first line <IfModule mod_deflate.c> <IfModule mod_filter.c> AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript AddOutputFilterByType DEFLATE application/rss+xml AddOutputFilterByType DEFLATE application/xml </IfModule> </IfModule> Restart the server to incorporate the changes. root@server:~# systemctl restart apache2.service == Install PHP == I install PHP in a moment for the [[Mediawiki farm setup]], but it needs tweaking first to allow HTTP2 to be enabled. * Install PHP and the FastCGI extension root@server:~# aptitude install php php-fpm php-apcu libapache2-mod-fastcgi * Enable PHP and the FastCGI modules root@server:~# a2enmod php7.2 root@server:~# a2enmod php7.2-fpm root@server:~# a2dismod php7.0 root@server:~# systemctl restart apache2 == Enable HTTP2 == To enable HTTP2, switch PHP to use the FPM module and Apache to the ''event'' handler. root@server:~# a2enmod proxy_fcgi root@server:~# a2enconf php7.2-fpm root@server:~# a2dismod mpm_prefork root@server:~# a2enmod mpm_event root@server:~# systemctl restart apache2 Test the PHP installation by creating a file <code>/var/www/site.domain.tld/test.php</code> containing <?php phpinfo(); ?> then visit that file in a browser. It should say Server API FPM/FastCGI near the top. Then enable HTTP2 by creating the file <code>/etc/apache2/conf-available/http2.conf</code> Protocols h2 http/1.1 # vim: syntax=apache ts=4 sw=4 sts=4 sr noet Then enable the module. root@server:~# a2enconf http2 root@server:~# a2enmod http2 root@server:~# systemctl restart apache2 Test it by pointing the [https://tools.keycdn.com/http2-test KeyCDN tester] at the domain. Once the FCGI plugin is working, allow it to use more workers by modifying <code>/etc/php/7.2/fpm/pool.d/www.conf</code> to update these lines (including commenting out <code>pm.start_servers</code>: pm.max_children = 5 ; pm.start_servers = 2 pm.min_spare_servers = 2 pm.max_spare_servers = 6 Then restart the FCGI server. root@server:~# systemctl restart php7.2-fpm.service The server should now be using all the CPU while serving many concurrent requests. == See also == Here are a few pages that are useful guides or provide background and context. * [http://httpd.apache.org/docs/2.2/vhosts/examples.html How to set up virtual hosts with Apache] * [https://help.ubuntu.com/community/forum/server/apache2/SSL SSL + Apache on Ubuntu] * [http://www.tc.umn.edu/~brams006/selfsign_ubuntu.html Another guide to SSL + Apache on Ubuntu] * [http://ubuntuforums.org/showpost.php?p=19832&postcount=4 A step-by-step guide to SSL + Apache on Ubuntu] * [http://www.securityfocus.com/infocus/1818 General guide to SSL + Apache] * [https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html Strong SSL Security on Apache2] * [http://www.howtoforge.com/apache2_mod_deflate How To Save Traffic With Apache2's mod_deflate] (Not discussed here: if you use the Deflate module, remember to enable it (a2enmod deflate) and create an empty log file (touch /var/log/apache2/project_deflate.log) before restarting Apache.) [[Category:Server setup]] 8ceaf01a993f86978295c68a1faf12e3e7fc7ba4 0 1540 2946 2888 2020-02-16T20:32:19Z Neil 2 /* Cron (and Anacron) timing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly Then restart the <code>cron</code> daemon. root@desktop:~# systemctl status cron.service [[Category:Server setup]] 8297394db975756f2a1a3621821a0cda0cb60800 2947 2946 2020-02-22T10:35:36Z Neil 2 /* Cron (and Anacron) timing */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly Then restart the <code>cron</code> daemon. root@desktop:~# systemctl restart cron.service === SystemD version === The timing of Anacron is controlled by SystemD, as specified in the <code>/etc/systemd/system/timers.target.wants/anacron.timer</code> file. Change the file to have the updated <code>OnCalendar</code> line: [Unit] Description=Trigger anacron every hour [Timer] # OnCalendar=*-*-* 07..23:30 OnCalendar=*-*-* 01:03 RandomizedDelaySec=5m Persistent=true [Install] WantedBy=timers.target Then restart the <code>anacron</code> daemon. root@desktop:~# systemctl restart anacron.service [[Category:Server setup]] a0c3969f0a0c69eea8d84ecc2f17ddbacc1a7134 2952 2947 2020-08-05T11:20:24Z Neil 2 /* Create users */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly Then restart the <code>cron</code> daemon. root@desktop:~# systemctl restart cron.service === SystemD version === The timing of Anacron is controlled by SystemD, as specified in the <code>/etc/systemd/system/timers.target.wants/anacron.timer</code> file. Change the file to have the updated <code>OnCalendar</code> line: [Unit] Description=Trigger anacron every hour [Timer] # OnCalendar=*-*-* 07..23:30 OnCalendar=*-*-* 01:03 RandomizedDelaySec=5m Persistent=true [Install] WantedBy=timers.target Then restart the <code>anacron</code> daemon. root@desktop:~# systemctl restart anacron.service [[Category:Server setup]] fd2a541e5dca1a60224f67eeb97ecd123ec91501 0 1493 2954 1641 2020-09-07T12:46:30Z Neil 2 /* Chaotics make for second-rate antagonists */ wikitext text/x-wiki People are arguing about how Chaos is perceived by Gloranthans. But I think it's useful to consider how Chaos is considered by us, the people that create Glorantha the game world. From that point of view, I think that Chaos is bad because Chaos is boring. The things that attracted me, as a player and GM, to Glorantha were its depth and the lack of easy moral decisions (Are the Lunars evil? Were the Romans evil?). This meant that I could imagine world as rich and complex as this one, and use it as a vehicle in which to tell engaging and satisfying stories. The presence of Chaos acts against this. There are several reasons for this. __TOC__ == Chaos is a cop-out for plot writers == It's very easy to write a plot with obvious "Good Guys" (the PCs) and "Bad Guys" (the chaotics). It gives the PCs a good motivation. Chaos has nifty "weird stuff" for a challenge. But doing so cheapens Glorantha. Good drama needs well-rounded characters, with both good and evil aspects to their personalities. They need complexity, and the ability to change and take on different roles at different times. Turning everything into two-dimensional "Good vs Evil" prevents that happening. Very often, conflicts are not caused by differences between Good and Evil, but simply by different sets of people having different agendas. Giving the opposition believable reasons for their actions allows me as a GM to run those characters better and more easily. This trend towards the "Good vs Evil" cop-out has been developing "recently". As far as I know, some 17% of Chaosium-published RQ2 scenarios revolved around chaos. Some 40% of Chaosium/AH-published RQ3 scenarios did. This is a dangerous trend, turning adventures into simple munchkin-friendly hack'n'slay sessions. Glorantha can be so much more than that. This trend might be reversed with the greater emphasis on socially-situated 'adventures' that might follow from the publication of Hero Wars and HeroQuest. I wait in hope. == Chaos means no moral or ethical decisions == The assumption that chaos = evil means that characters, and therefore players, never have to consider the ethics of their actions. For example, at the end of Gamauta's Vision, the PCs have to decide what to do with a bunch of chaotic children. An interesting moral question. Lots of MGF there as the PCs explore and argue the options. At least, there would be if it wasn't obvious to the PCs that chaos = evil, therefore the children must be killed. At the beginning of the Troubled Waters campaign, the PCs had real qualms about slaughtering the cute, innocent, little mudsharks. At the end of the campaign, they had no problem killing cute, innocent, little chaos monstrosities. Guess which episode they talk about more? And removing the big "Beware: Evil! Soul Destruction in Progress!" sign from cults like Thanatar makes them more appealing, more seductive, more interesting. Humakt offers great power, but at a cost to society. Thanatar offers great power, but at a cost to society. Is one better than the other? Are there situations where gaining Thanatari or Humakti powers are justified? Does the end ever justify the means? Without chaos, you can ask this question. == Chaos makes characters lack depth == If the focus of a campaign is simply killing chaos, all the PCs will be concerned with is killing chaos. The characters will be one-dimensional. Their opponents will be chaos, and exist simply to be killed (eventually). Also one-dimensional. But give all the characters, PCs and NPCs, more objectives, more avenues, and you'll get a more rewarding and enjoyable game. == Chaotic villains are treated as victims == At present, players have no reason to sympathise with their enemies. If a character is evil, does evil things, it's not their fault. They're chaotic: they can't help it. There's no sense of people being responsible for their actions, and the effects they have. Nobody has to think why evil happens, how it can be stopped, what role they play in bringing it about. == Chaotics make for second-rate antagonists == :"Humans are the only race more dangerous that broos in the [Zola Fel] Valley" :(RoC, p. 109) If I want to give my players a run for their money, why should I give them second-rate opposition? Why not set them against the most dangerous opposition there is? In fact, I think that the presence of Chaos in Glorantha is so pernicious that I think it should be removed from all our contacts with the world. If we do that, what do we loose? Very little, if anything. * Illumination remains, as a theist version of mysticism. It would have exactly the same game effects. * Storm Bulls remain psychopathic killers, but instead of being anti-chaos, they become anti-everyone. This raises more interesting questions for their clan leaders, increasing MGF and leading to a better game. * The Orlanthi will still hate the Lunars: the Lunars have invaded, are changing the Orlanthi way of life, changing the mythic landscape. * "Chaos" gods remain hated or feared. Thanatari still steal knowledge, Krarshti still conspire, Thedi still rape. Humakti still kill. * The lesser and greater darkness from the godtime remain, but are the same thing. Very few of the myths change. * The Compromise remains, to include and control the changes of the Storm Age into the stability of the Golden Age. The real world doesn't have clearly labelled evil. This makes for difficult judgements. Gameworlds like ''Call of Cthulhu'' don't have clearly labelled evil. That makes for interesting play. Why should Glorantha be so impoverished? (For some related thoughts on morality and RPGs, have a look at John Tynes's game [http://johntynes.com/revland2000/rl_powerkill.html Power Kill].) [[Category:Glorantha]] 0ae9df8535c6188dc8dfd23092ac2780bf0e12b8 0 1532 2958 2882 2020-11-13T09:03:48Z Neil 2 /* Create virtual users */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.roundcube.net/ Roundcube] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Roundcube setup is described in the [[Webmail setup]] page, but requires the [[Web server setup]], including HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap ==== Note to self: don't disable to protocol, turn them off in 10-master.conf ==== * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. ==== Note to self: also disable pop3, as below: ==== service pop3-login { inet_listener pop3 { #port = 110 port = 0 } inet_listener pop3s { #port = 995 #ssl = yes port = 0 } } * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl = yes ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth test user@domain.tld mailpassword root@server:~# doveadm auth login user@domain.tld mailpassword * Create a separate mailbox for <code>postmaster</code>, even if it delivers mail to the same virtual mail directory as a user. You'll need this for accepting spam reports from Amavis, which you'll set up next. * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] 0b7e76e89453f9005205511013b22e8918d583b6 2965 2958 2021-02-08T15:00:24Z Neil 2 /* Note to self: also disable pop3, as below: */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.roundcube.net/ Roundcube] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Roundcube setup is described in the [[Webmail setup]] page, but requires the [[Web server setup]], including HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap ==== Note to self: don't disable to protocol, turn them off in 10-master.conf ==== * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. ==== Note to self: also disable pop3, as below: ==== service pop3-login { inet_listener pop3 { #port = 110 port = 0 } inet_listener pop3s { #port = 995 #ssl = yes port = 0 } } * Enable longer DH keys by running (it may take a while): root@server:/etc/dovecot# openssl dhparam -out /etc/dovecot/dh.pem 4096 * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl = yes ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem ssl_dh = </etc/dovecot/dh.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth test user@domain.tld mailpassword root@server:~# doveadm auth login user@domain.tld mailpassword * Create a separate mailbox for <code>postmaster</code>, even if it delivers mail to the same virtual mail directory as a user. You'll need this for accepting spam reports from Amavis, which you'll set up next. * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] a999827294c239bb23ed291f85c37d187da7b7ed 2966 2965 2021-02-09T09:35:21Z Neil 2 /* Initial Dovecot configuration */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |align="center" width="50%"| [[Postfix server setup|MTA (Postfix)]] |} Configuring email is the most involved part of setting up a server. My setup is organised with: * [http://www.postfix.org Postfix] as a mail transfer agent, using virtual users with maildir-style mailboxes, and TLS transport * [http://spamassassin.apache.org/ SpamAssassin] for spam filtering * [http://www.clamav.net/ ClamAV] for virus filtering * [http://www.amavis.org/ Amavis] to wrap SpamAssassin and ClamAV for easy use with Postfix * [http://wiki2.dovecot.org/FrontPage Dovecot] as a mail delivery agent, providing IMAP services * Dovecot's [http://wiki2.dovecot.org/LDA local delivery agent], to do server-side filtering with [http://sieve.info/ Sieve] * [http://www.roundcube.net/ Roundcube] to provide a webmail, running over a secure web connection A lot to do, and split between several pages on this site. Dovecot 2, Dovecot LDA, and Sieve setup are on here. Postfix, TLS, and Amavis + SpamAssassin + ClamAV are on the [[Postfix server setup]] page. Roundcube setup is described in the [[Webmail setup]] page, but requires the [[Web server setup]], including HTTPS configuration. We set up Dovecot first as it provides authentication and local delivery services for Postfix. However, there will still be a lot of flipping between the two. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. However, those articles still refer to Dovecot 1, while Ubuntu 12.04 has Dovecot 2. The configuration in the two packages is very different: Dovecot 1 has a single config file, while Dovecot 2 uses a directory of small config files. == Virtual user structure == I use virtual users because I handle mail for many users in many domains, and want the flexibility that virtual users allows. Information about the virtual users is held in plain text files, as that's all I need for the small number of users I have. The virtual user mailboxes are in the directory structure /var/vmail/domain1.com/user1/mail/ /user2/mail/ /domain2.com/user1/mail/ /user2/mail/ /domain3.com/user1/mail/ ...and so on. The <code>/var/vmail/domain.com/user1/</code> directory will be used to store configuration files (such as the Sieve files) for each user, with the <code>/var/vmail/domain.com/user1/mail/</code> directory being the base Maildir directory for the user. The <code>vmail</code> user will be the one used for email transport into and out of these virtual mailboxes. == Initial Dovecot configuration == This gets a basic IMAP server up and running. * Install Dovecot root@server:~# aptitude install dovecot dovecot-imapd whois dovecot-sieve dovecot-managesieved :(<code>whois</code> is needed in a moment for creating passwords) * Create the <code>vmail</code> user and group root@server:~# groupadd -g 5000 vmail root@server:~# useradd -m -u 5000 -g 5000 -s /bin/bash vmail * Modify the protocols in <code>/etc/dovecot/dovecot.conf</code> to read: # Enable installed protocols # !include_try /usr/share/dovecot/protocols.d/*.protocol protocols = imap * My installation seemed to require the creation of <code>/etc/dovecot/conf.d/20-stats.conf</code> service stats { unix_listener stats-reader { user = vmail group = vmail mode = 0660 } unix_listener stats-writer { user = vmail group = vmail mode = 0660 } } ==== Note to self: don't disable to protocol, turn them off in 10-master.conf ==== * Modify <code>/etc/dovecot/conf.d/10-mail.conf</code> to contain: #mail_location = mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail namespace inbox { prefix = inbox = yes hidden = no subscriptions = yes } * Modify <code>/etc/dovecot/conf.d/10-auth.conf</code> to contain: disable_plaintext_auth = yes auth_mechanisms = plain #!include auth-system.conf.ext !include auth-passwdfile.conf.ext :The first line means that all connections to Dovecot must be encrypted (apart from connections from the same machine). There's no need to specify separate IMAP and IMAPS protocols. * Modify <code>/etc/dovecot/conf.d/10-master.conf</code> to contain the <code>port = 0</code> line: service imap-login { inet_listener imap { #port = 143 } inet_listener imaps { #port = 993 #ssl = yes port = 0 # disable this listener } :This disables the login listener from listening on port 993. ==== Note to self: also disable pop3, as below: ==== service pop3-login { inet_listener pop3 { #port = 110 port = 0 } inet_listener pop3s { #port = 995 #ssl = yes port = 0 } } * Enable longer DH keys by running (it may take a while): root@server:/etc/dovecot# openssl dhparam -out /etc/dovecot/dh.pem 4096 * Modify <code>/etc/dovecot/conf.d/10-ssl.conf</code> to contain: ssl = yes ssl_cert = </etc/letsencrypt/live/imap.domain.tld/fullchain.pem ssl_key = </etc/letsencrypt/live/imap.domain.tld/privkey.pem ssl_dh = </etc/dovecot/dh.pem == Create virtual users == * Modify <code>/etc/dovecot/conf.d/auth-passwdfile.conf.ext</code> to contain: passdb { driver = passwd-file args = scheme=plain-md5 username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users } This shows that the user information (username, password, domain name, and vmail directory) is contained in <code>/etc/dovecot/users</code>. This is in the same format as <code>/etc/passwd</code>. It's easier to create a script to add the users to Dovecot, and tell Postfix about the maps. * Create the script <code>/etc/dovecot/dovecot-adduser</code> #!/bin/bash username=${1%%@*} domain=${1#*@} # create the user name in the user file echo "$username@$domain:`mkpasswd --hash=md5 $2`:5000:5000::/var/vmail/$domain/$username/::" >> /etc/dovecot/users # create the maildir directory structure /usr/bin/maildirmake.dovecot /var/vmail/$domain/$username/mail 5000:5000 chown -R vmail:vmail /var/vmail/$domain # add the user to the Postfix virtual map file echo $1 $domain/$username/mail/ >> /etc/postfix/vmaps postmap /etc/postfix/vmaps postfix reload :Make sure the script is executable. * Create the users with <code>dovecot-adduser</code>, then check that it works: root@server:~# /etc/dovecot/dovecot-adduser user@domain.tld mailpassword root@server:~# doveadm auth test user@domain.tld mailpassword root@server:~# doveadm auth login user@domain.tld mailpassword * Create a separate mailbox for <code>postmaster</code>, even if it delivers mail to the same virtual mail directory as a user. You'll need this for accepting spam reports from Amavis, which you'll set up next. * Once every virtual user has received at least one mail (and hence had their mailbox directory structure created), change the file ownerships on those directories so that Dovecot can manipulate them root@server:~# chown -R vmail:vmail /home/vmail == Testing Dovecot == Type in a terminal root@server:~# telnet mail.domain1.com 143 An output like the following will display in your terminal Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. Type the following code segment in the prompt provided by the Dovecot IMAP server. a login user1@domain1.com password a logout If you can log in, Final output should be something like this: Trying 69.60.109.217... Connected to mail.domain1.com. Escape character is '^]'. +OK dovecot ready. a login info@domain1.com password a OK Logged in. a logout * BYE Logging out a OK Logout completed. You should now be able to log into the Dovecot server with your favourite email client. IMAPS connections can be tested with <code>openssl</code> root@server:~# openssl s_client -connect imap.example.com:143 -starttls imap <verbiage snipped> . OK Pre-login capabilities listed, post-login capabilities have more. :and test the login as before. Logout with a logout Mail clients should connect to <code>imap.domain.tld:143</code> using SSL encryption and PLAIN authentication. Once IMAP and users are set up, you can use that as the basis for SASL authentication of SMTP users of [[Postfix server setup|Postfix]]. See the [[Postfix server setup#Allowing off-site users with SASL|Postfix SASL notes]] for details. ==Logging== You now need to ensure that the Dovecot logs to its own log files. (The Dovecot documentation recommends using <code>rsyslog</code>, but I couldn't get that to work.) * Modify <code>/etc/dovecot/conf.d/10-logging.conf</code> to contain: log_path = /var/log/dovecot.log # syslog_facility = local1 * Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> to contain the <code>log_path</code>: protocol lda { mail_plugins = $mail_plugins sieve log_path = /var/log/dovecot-lda.log postmaster_address = postmaster@domain.tld } * Make <code>/var/log/dovecot-lda.log</code> world-writable: root@server:~# touch /var/log/dovecot-lda.log root@server:~# chmod a+w /var/log/dovecot-lda.log * Reload the Dovecot configuration and check it's picked up the logfile locations: root@server:~# service dovecot reload root@server:~# doveadm log find Debug: /var/log/dovecot.log Info: /var/log/dovecot.log Warning: /var/log/dovecot.log Error: /var/log/dovecot.log Fatal: /var/log/dovecot.log * Create <code>/etc/logrotate.d/dovecot</code> to contain: /var/log/dovecot.log { weekly rotate 52 missingok notifempty compress delaycompress create 640 root adm sharedscripts postrotate doveadm log reopen endscript } /var/log/dovecot-lda.log { weekly rotate 52 missingok notifempty compress delaycompress create 666 root adm sharedscripts postrotate doveadm log reopen endscript } :The two separate sections are so that the <code>dovecot-lda.log</code> file remains world-writable. == Set up local delivery and sieve == The final stage is to incorporate Dovecot's local delivery agent, [http://wiki2.dovecot.org/LDA LDA], into the mail delivery system. I want to do this because Deliver supports [http://sieve.info/ Sieve], which does server-side mail filtering. See the [http://wiki2.dovecot.org/Pigeonhole/Sieve Dovecot Sieve] page for details. Modify <code>/etc/dovecot/conf.d/15-lda.conf</code> recipient_delimiter = . lda_mailbox_autocreate = yes protocol lda { mail_plugins = $mail_plugins sieve } * Create an an empty log file for <code>deliver</code> root@server:~# touch /var/log/dovecot-deliver.log root@server:~# chmod a+w /var/log/dovecot-deliver.log * Ensure the changes are incorporated into Postfix and Dovecot root@server:~# postfix reload root@server:~# /etc/init.d/dovecot restart * Now, create rule files for Deliver/Sieve to use. This should be in the base directory for the user's mail directory, e.g. <code>/home/vmail/domain1.com/user1/.dovecot.sieve</code> require ["fileinto"]; # generally a clause is # if header :contains ["From", "To", "CC", "Subject", <and so on>] # ["part of address", "another string element", <and so on> # If any of the elements match any of the header parts, the clause is triggered if header :contains ["To", "CC"] ["user1@domain1.com", "user2@domain2.com"] { fileinto "Folder 1"; } elsif header :contains ["From"] ["person@somewhere.com"] { fileinto "Folder 1.Subfolder 2"; } * Ensure that all the folders referenced in the rules exist before you mention them in the <code>.dovecot.sieve</code> file. The first time Dovecot delivers mail to this user, it will create the <code>.dovecot.sievec</code> compiled file. If the rules file changes, Dovecot will recreate the compiled file automatically. For more on Sieve rules, see the [http://wiki.fastmail.fm/index.php?title=Main_Page FastMail wiki] which has a [http://wiki.fastmail.fm/index.php?title=BasicSieve guide to basic Sieve] and [http://wiki.fastmail.fm/index.php?title=SieveExamples some examples], and a [http://www.elvey.com/it/sieve/ Sieve summary], including a [http://www.elvey.com/it/sieve/SieveScript.txt comprehensive Sieve rule set]. And that should be email sorted out. == Troubleshooting == ===Dovecot doesn't start=== If Dovecot fails to start and <code>/var/log/dovecot.log</code> contains lines like dovecot: IMAP(user): FETCH for mailbox INBOX UID 176705 failed to read message input: Is a directory it means that the Mbox folders for the mail folder in question (in this case INBOX, but it could be anything) contain directories instead of files. Delete the directories from <code>/home/vmail/domain/user/INBOX/cur</code> and restart Dovecot. (Thanks to [http://notjustlinux.blogspot.com/2009/05/dovecot-migration-error.html Patrick Bulteel] for the pointer.) ===Mail folder sbscriptions=== Don't forget that users will need to subscribe to the folders they want to see. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.dovecot.org/list/dovecot/2006-September/015955.html dovecot-sieve with FreeBSD] * [http://wiki.dovecot.org/LDA Dovecot LDA] * [http://www.ducea.com/2006/06/06/rotating-linux-log-files-part-2-logrotate/ Rotating log files] * [http://www.credentia.cc/certs/howto/dovecot.html Enabling Secure IMAP and POP3 with Dovecot] * [http://www.knowplace.org/pages/howtos/linux_imapd_with_ssl_howto.php Linux imapd with SSL Howto] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [https://wiki.syscp.org//contrib/Dovecot Setting up Dovecot to use Postfix and Deliver] [[Category:Server setup]] d0fbe56ffb783546f6e8bc54cffd10cf136ca5d4 0 1531 2959 2956 2020-11-13T09:06:01Z Neil 2 /* Amavis */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/50_user_defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] fdde6e73744b3db7ef7cd73ebece48f789a60cb3 2960 2959 2020-11-13T09:07:34Z Neil 2 /* Postfix */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/50_user_defaults</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 7e2784085bc9cd3118af0fdbf59da860f6a23777 2961 2960 2020-11-13T09:32:04Z Neil 2 /* Amavis */ Added detail on quarantine wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/50-user</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service * Once Amavis is working and correctly identifying spam and malware, you can adjust the settings in <code>/etc/amavis/conf.d/50-user</code> to classify more spam, and reject the most spammy. (Rejected messages are quarantined, with a notification going to <code>postmaster@domain.tld</code>.) $enable_dkim_verification = 1; # was disabled to prevent warning $sa_tag_level_deflt = 0.0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 1.5; # add 'spam detected' headers at that level $sa_kill_level_deflt = 2.5; # triggers spam evasive actions $sa_dsn_cutoff_level = 6.0; # spam level beyond which a DSN is not sent, was 10 $final_bad_header_destiny = D_PASS; # False-positive prone (for spam) $final_virus_destiny = D_DISCARD; # (data not lost, see virus quarantine) $final_banned_destiny = D_REJECT; # D_REJECT when front-end MTA $final_spam_destiny = D_DISCARD; # Where quarantine notifications go $virus_admin = "postmaster\@$mydomain"; # due to D_DISCARD default $spam_admin = "postmaster\@$mydomain"; # due to D_DISCARD default # Where quarantine notifications appear to come from $mailfrom_notify_admin = "contentfilter\@$mydomain"; $mailfrom_notify_recip = $mailfrom_notify_admin; $mailfrom_notify_spamadmin = $mailfrom_notify_admin; $hdrfrom_notify_sender = $mailfrom_notify_admin; $hdrfrom_notify_admin = $mailfrom_notify_admin; Spam will be quarantined and notifications sent to <code>postmaster@domain.tld</code>. The notification will contain lines like: Content type: Spam Internal reference code for the message is 26829-01/ZKBMjq6S2Dsi ... The message has been quarantined as: Z/spam-ZKBMjq6S2Dsi.gz The message WAS NOT relayed to: <someone@domain.tld>: 250 2.7.0 Ok, discarded, id=26829-01 - spam You can release the quarantined message with <code>amavisd-release</code>, like so: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz or send it message to another user by giving their address: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz admin@domain.tld Delete old spam by creating the file <code>/etc/cron.daily/delete_old_spam</code> containing #!/bin/sh find /var/lib/amavis/virusmails/ -type f -mtime +31 -delete and make it executable root@server:~# chmod u+x /etc/cron.daily/delete_old_spam === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 0f3af16b8c46b30f87305fadfa026f983e29860a 2998 2961 2022-07-23T16:26:47Z Neil 2 /* DKIM */ ipv6 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/50-user</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service * Once Amavis is working and correctly identifying spam and malware, you can adjust the settings in <code>/etc/amavis/conf.d/50-user</code> to classify more spam, and reject the most spammy. (Rejected messages are quarantined, with a notification going to <code>postmaster@domain.tld</code>.) $enable_dkim_verification = 1; # was disabled to prevent warning $sa_tag_level_deflt = 0.0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 1.5; # add 'spam detected' headers at that level $sa_kill_level_deflt = 2.5; # triggers spam evasive actions $sa_dsn_cutoff_level = 6.0; # spam level beyond which a DSN is not sent, was 10 $final_bad_header_destiny = D_PASS; # False-positive prone (for spam) $final_virus_destiny = D_DISCARD; # (data not lost, see virus quarantine) $final_banned_destiny = D_REJECT; # D_REJECT when front-end MTA $final_spam_destiny = D_DISCARD; # Where quarantine notifications go $virus_admin = "postmaster\@$mydomain"; # due to D_DISCARD default $spam_admin = "postmaster\@$mydomain"; # due to D_DISCARD default # Where quarantine notifications appear to come from $mailfrom_notify_admin = "contentfilter\@$mydomain"; $mailfrom_notify_recip = $mailfrom_notify_admin; $mailfrom_notify_spamadmin = $mailfrom_notify_admin; $hdrfrom_notify_sender = $mailfrom_notify_admin; $hdrfrom_notify_admin = $mailfrom_notify_admin; Spam will be quarantined and notifications sent to <code>postmaster@domain.tld</code>. The notification will contain lines like: Content type: Spam Internal reference code for the message is 26829-01/ZKBMjq6S2Dsi ... The message has been quarantined as: Z/spam-ZKBMjq6S2Dsi.gz The message WAS NOT relayed to: <someone@domain.tld>: 250 2.7.0 Ok, discarded, id=26829-01 - spam You can release the quarantined message with <code>amavisd-release</code>, like so: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz or send it message to another user by giving their address: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz admin@domain.tld Delete old spam by creating the file <code>/etc/cron.daily/delete_old_spam</code> containing #!/bin/sh find /var/lib/amavis/virusmails/ -type f -mtime +31 -delete and make it executable root@server:~# chmod u+x /etc/cron.daily/delete_old_spam === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 aaaa:bbbb:cccc:dddd/64 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 7c5cd235eee7e4e244acc0a9da85f45382677f08 2999 2998 2022-07-23T16:27:52Z Neil 2 /* DMARC */ ipv6 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/50-user</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service * Once Amavis is working and correctly identifying spam and malware, you can adjust the settings in <code>/etc/amavis/conf.d/50-user</code> to classify more spam, and reject the most spammy. (Rejected messages are quarantined, with a notification going to <code>postmaster@domain.tld</code>.) $enable_dkim_verification = 1; # was disabled to prevent warning $sa_tag_level_deflt = 0.0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 1.5; # add 'spam detected' headers at that level $sa_kill_level_deflt = 2.5; # triggers spam evasive actions $sa_dsn_cutoff_level = 6.0; # spam level beyond which a DSN is not sent, was 10 $final_bad_header_destiny = D_PASS; # False-positive prone (for spam) $final_virus_destiny = D_DISCARD; # (data not lost, see virus quarantine) $final_banned_destiny = D_REJECT; # D_REJECT when front-end MTA $final_spam_destiny = D_DISCARD; # Where quarantine notifications go $virus_admin = "postmaster\@$mydomain"; # due to D_DISCARD default $spam_admin = "postmaster\@$mydomain"; # due to D_DISCARD default # Where quarantine notifications appear to come from $mailfrom_notify_admin = "contentfilter\@$mydomain"; $mailfrom_notify_recip = $mailfrom_notify_admin; $mailfrom_notify_spamadmin = $mailfrom_notify_admin; $hdrfrom_notify_sender = $mailfrom_notify_admin; $hdrfrom_notify_admin = $mailfrom_notify_admin; Spam will be quarantined and notifications sent to <code>postmaster@domain.tld</code>. The notification will contain lines like: Content type: Spam Internal reference code for the message is 26829-01/ZKBMjq6S2Dsi ... The message has been quarantined as: Z/spam-ZKBMjq6S2Dsi.gz The message WAS NOT relayed to: <someone@domain.tld>: 250 2.7.0 Ok, discarded, id=26829-01 - spam You can release the quarantined message with <code>amavisd-release</code>, like so: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz or send it message to another user by giving their address: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz admin@domain.tld Delete old spam by creating the file <code>/etc/cron.daily/delete_old_spam</code> containing #!/bin/sh find /var/lib/amavis/virusmails/ -type f -mtime +31 -delete and make it executable root@server:~# chmod u+x /etc/cron.daily/delete_old_spam === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 aaaa:bbbb:cccc:dddd/64 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 aaaa:bbbb:cccc:dddd::/64 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 09292fc7686ae220960ef447755f69417b9ae157 3006 2999 2023-01-12T10:31:29Z Neil 2 /* Basic installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix which IPv6 address it should use when sending messages. Add the line <code>-o smtp_bind_address6</code> below the <code>smtp</code> statement in <code>master.cf</code> smtp unix - - y - - smtp -o smtp_bind_address6=1:2:3::25 * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/50-user</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service * Once Amavis is working and correctly identifying spam and malware, you can adjust the settings in <code>/etc/amavis/conf.d/50-user</code> to classify more spam, and reject the most spammy. (Rejected messages are quarantined, with a notification going to <code>postmaster@domain.tld</code>.) $enable_dkim_verification = 1; # was disabled to prevent warning $sa_tag_level_deflt = 0.0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 1.5; # add 'spam detected' headers at that level $sa_kill_level_deflt = 2.5; # triggers spam evasive actions $sa_dsn_cutoff_level = 6.0; # spam level beyond which a DSN is not sent, was 10 $final_bad_header_destiny = D_PASS; # False-positive prone (for spam) $final_virus_destiny = D_DISCARD; # (data not lost, see virus quarantine) $final_banned_destiny = D_REJECT; # D_REJECT when front-end MTA $final_spam_destiny = D_DISCARD; # Where quarantine notifications go $virus_admin = "postmaster\@$mydomain"; # due to D_DISCARD default $spam_admin = "postmaster\@$mydomain"; # due to D_DISCARD default # Where quarantine notifications appear to come from $mailfrom_notify_admin = "contentfilter\@$mydomain"; $mailfrom_notify_recip = $mailfrom_notify_admin; $mailfrom_notify_spamadmin = $mailfrom_notify_admin; $hdrfrom_notify_sender = $mailfrom_notify_admin; $hdrfrom_notify_admin = $mailfrom_notify_admin; Spam will be quarantined and notifications sent to <code>postmaster@domain.tld</code>. The notification will contain lines like: Content type: Spam Internal reference code for the message is 26829-01/ZKBMjq6S2Dsi ... The message has been quarantined as: Z/spam-ZKBMjq6S2Dsi.gz The message WAS NOT relayed to: <someone@domain.tld>: 250 2.7.0 Ok, discarded, id=26829-01 - spam You can release the quarantined message with <code>amavisd-release</code>, like so: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz or send it message to another user by giving their address: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz admin@domain.tld Delete old spam by creating the file <code>/etc/cron.daily/delete_old_spam</code> containing #!/bin/sh find /var/lib/amavis/virusmails/ -type f -mtime +31 -delete and make it executable root@server:~# chmod u+x /etc/cron.daily/delete_old_spam === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 aaaa:bbbb:cccc:dddd/64 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 aaaa:bbbb:cccc:dddd::/64 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 350e2dd24c86c7921d01f325a7fd908f0483693d 3007 3006 2023-08-08T14:22:17Z Neil 2 /* DKIM */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix which IPv6 address it should use when sending messages. Add the line <code>-o smtp_bind_address6</code> below the <code>smtp</code> statement in <code>master.cf</code> smtp unix - - y - - smtp -o smtp_bind_address6=1:2:3::25 * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/50-user</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service * Once Amavis is working and correctly identifying spam and malware, you can adjust the settings in <code>/etc/amavis/conf.d/50-user</code> to classify more spam, and reject the most spammy. (Rejected messages are quarantined, with a notification going to <code>postmaster@domain.tld</code>.) $enable_dkim_verification = 1; # was disabled to prevent warning $sa_tag_level_deflt = 0.0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 1.5; # add 'spam detected' headers at that level $sa_kill_level_deflt = 2.5; # triggers spam evasive actions $sa_dsn_cutoff_level = 6.0; # spam level beyond which a DSN is not sent, was 10 $final_bad_header_destiny = D_PASS; # False-positive prone (for spam) $final_virus_destiny = D_DISCARD; # (data not lost, see virus quarantine) $final_banned_destiny = D_REJECT; # D_REJECT when front-end MTA $final_spam_destiny = D_DISCARD; # Where quarantine notifications go $virus_admin = "postmaster\@$mydomain"; # due to D_DISCARD default $spam_admin = "postmaster\@$mydomain"; # due to D_DISCARD default # Where quarantine notifications appear to come from $mailfrom_notify_admin = "contentfilter\@$mydomain"; $mailfrom_notify_recip = $mailfrom_notify_admin; $mailfrom_notify_spamadmin = $mailfrom_notify_admin; $hdrfrom_notify_sender = $mailfrom_notify_admin; $hdrfrom_notify_admin = $mailfrom_notify_admin; Spam will be quarantined and notifications sent to <code>postmaster@domain.tld</code>. The notification will contain lines like: Content type: Spam Internal reference code for the message is 26829-01/ZKBMjq6S2Dsi ... The message has been quarantined as: Z/spam-ZKBMjq6S2Dsi.gz The message WAS NOT relayed to: <someone@domain.tld>: 250 2.7.0 Ok, discarded, id=26829-01 - spam You can release the quarantined message with <code>amavisd-release</code>, like so: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz or send it message to another user by giving their address: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz admin@domain.tld Delete old spam by creating the file <code>/etc/cron.daily/delete_old_spam</code> containing #!/bin/sh find /var/lib/amavis/virusmails/ -type f -mtime +31 -delete and make it executable root@server:~# chmod u+x /etc/cron.daily/delete_old_spam === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 aaaa:bbbb:cccc:dddd/64 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * If needed, add the entry to <code>/etc/opendkim/key_table</code>: mail._domainkey.domain.tld domain.tld:mail:/etc/opendkim/domain.tld/mail.private * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 aaaa:bbbb:cccc:dddd::/64 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 257e939aab29193e8bd37758a10b66eaaa0313c7 3008 3007 2023-10-19T09:26:18Z Neil 2 Added Spamhaus block list instructions wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |align="center" width="50%"| [[Desktop mail forwarding]] |} This step of the howto assumes you've already set up [[Dovecot server setup|Dovecot]] to provide authentication and local delivery services. This page describes the setup for Postfix, TLS, and Amavis + SpamAssassin + ClamAV. Dovecot, and Sieve setup are on the [[Dovecot server setup]] page. Squirrelmail setup is described in the [[Web server setup]] page, along with the HTTPS configuration. The basic sources for setting up Postfix and Dovecot are the [https://help.ubuntu.com/community/PostfixBasicSetupHowto basic Postfix howto] and the [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto Postfix virtual user howto] on the Ubuntu wiki. ==Basic installation== * Install postfix root@server:~# aptitude install postfix mailx * Update <code>/etc/postfix/main.cf</code> compatibility_level = 2 mydomain = domain1.com # Change this line for your domain myorigin = $mydomain myhostname = mail.$mydomain # Change this for your mailserver's name mydestination = $myhostname localhost.$mydomain localhost relaydomains = $mydomain relayhost = mynetworks = 127.0.0.0/8 192.168.1.0/24 smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination ## Anti-spammer measures # Don't talk to mail systems that don't know their own hostname. smtpd_helo_restrictions = reject_unknown_helo_hostname # Don't accept mail from domains that don't exist. smtpd_sender_restrictions = reject_unknown_sender_domain # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining proxy_interfaces = 1.2.3.4 # My external IP smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mailbox_size_limit = 0 recipient_delimiter = .+ # Make sure you change this from a '+' inet_interfaces = all home_mailbox = Maildir/ virtual_mailbox_domains = /etc/postfix/vhosts virtual_mailbox_base = /var/vmail virtual_mailbox_maps = hash:/etc/postfix/vmaps virtual_minimum_uid = 1000 virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = hash:/etc/postfix/valiases * Tell Postfix which IPv6 address it should use when sending messages. Add the line <code>-o smtp_bind_address6</code> below the <code>smtp</code> statement in <code>master.cf</code> smtp unix - - y - - smtp -o smtp_bind_address6=1:2:3::25 * Tell Postfix about the domains it should service by creating <code>/etc/postfix/vhosts</code> domain1.com domain2.com * Tell Postfix about the users in those domains by creating <code>/etc/postfix/vmaps</code> user1@domain1.com domain1.com/user1/mail/ user2@domain1.com domain1.com/user2/mail/ user1@domain2.com domain2.com/user1/mail/ user3@domain2.com domain2.com/user3/mail/ :Then create the hash file: root@server:~# postmap /etc/postfix/vmaps :(this command needs to be repeated every time <code>/etc/postfix/vmaps</code> is changed) * Create aliases for the users in <code>/etc/postfix/valiases</code> root@domain1.com user1@domain1.com postmaster@domain1.com user1@domain1.com postmaster@domain2.com user1@domain1.com user2@domain1.com user2@domain2.com root@server.domain1.com user1@domain1.com root@desktop.domain1.com user1@domain1.com :Then create the hash file: root@server:~# postmap /etc/postfix/valiases :(this command needs to be repeated every time <code>/etc/postfix/valiases</code> is changed) * Create <code>/etc/aliases</code> root: user1 clamav: root amavis: root postmaster: root : and then create the database root@server:~# newaliases * The virtual mailbox directory structure should be created for each user when they receive their first email. * Get Postfix to reload the updated configuration files root@server:~#postfix reload * Test that Postfix is running properly. First, use <code>telnet</code> to connect to postfix root@server:~# telnet localhost 25 :Postfix will respond Trying 127.0.0.1... Connected to mail.server.org. Escape character is '^]'. 220 localhost.localdomain ESMTP Postfix (Ubuntu) :then type the following ehlo localhost mail from: root@localhost rcpt to: user1@domain1.com data Subject: My first mail on Postfix Hi, Are you there? regards, Admin . (Type the .[dot] in a new Line and press Enter ) quit :then check that the message is delivered properly into the correct mailbox root@server:~# ls /home/vmail/domain1/user1 * Then test that all the mail transport is working, using the <code>mail</code> command line utility root@server:~# mail user2@domain1.com :and again, check that the mail appears in the filesystem. == Adding TLS == Now to enable TLS, which encrypts communication between MTAs, preventing eavesdropping. We'll use the [[SSL certificate generation|certificates generated previously]]. All that's needed is to modify the <code>/etc/postfix/main.cf</code> file to point to the certificates. # TLS parameters smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.domain.tld/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/webmail.domain.tld/privkey.pem # smtpd_use_tls=yes smtpd_tls_security_level=may smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache and then reload Postfix root@server:~# postfix reload Check it's working by telnetting into Postfix root@server:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 mail.domain.tld ESMTP Postfix (Ubuntu) ehlo localhost and check that the line 250-STARTTLS appears in the output. Use <code>quit</code> to end the session. Clients should connect on port 25, using TLS security. == Allowing off-site users with SASL == As it stands, the server will only forward mail sent from the LAN. That's fine, but if you want to send mail from other devices via this SMTP server, you need to put some authentication on it. SASL is the general term for authentication on mail servers. Assuming you have [[Dovecot server setup|Dovecot]] set up with some users (up to the '[[Dovecot server setup#Testing_Dovecot|Testing Dovecot]]' stage, you can use Dovecot's authentication engine for authenticating users of the SMTP server. First, extend the Dovecot configuration file <code>/etc/dovecot/conf.d/10-master.conf</code> with the <code>unix_listener</code> section below so that it accepts authentication requests from Postfix. service auth { unix_listener auth-userdb { mode = 0660 user = vmail group = vmail } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { group = postfix user = postfix mode = 0666 } # Auth process is run as this user. #user = $default_internal_user } service auth-worker { # Auth worker process is run as root by default, so that it can access # /etc/shadow. If this isn't necessary, the user should be changed to # $default_internal_user. #user = root user = $default_internal_user } Next, make Postfix authenticate users. Modify <code>/etc/postfix/main.cf</code> to include the lines smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_recipient_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache Restart both Dovecot and Postfix: root@server:~# service dovecot reload root@server:~# postfix reload Mail clients can now post messages through the SMTP server, so long as they give a valid username/password combination that's recognised by Dovecot. == Amavis + ClamAV + SpamAssassin == Both ClamAV and SpamAssassin need to be introduced into Postfix's message-handlng queue to do their things with messages. The idea is that Postfix will take the message out of it's queue and hand it over the filter program (listening on the loopback interface), who will return it the same way after doing its thing. The trouble is, Postfix can only have one such filter, so Amavis is needed to act as a wrapper around them both. Here is how to do it. For instructions, I basically followed [https://help.ubuntu.com/community/PostfixAmavisNew the ones on the Ubuntu wiki]. * Create the file <code>/etc/mailname</code> containing mydomain.com * Install the packages: root@server:~# aptitude install amavisd-new spamassassin clamav-daemon pyzor razor libnet-dns-perl libmail-spf-perl root@server:~# aptitude install arj bzip2 cabextract cpio file gzip lhasa nomarch pax rar unrar unzip zip zoo root@server:~# aptitude install lrzip liblz4-tool rpm2cpio unrar-free ripole p7zip-full p7zip-rar lzop === ClamAV === * Add the ClamAV user to the Amavis group, and vice versa root@server:~# adduser clamav amavis root@server:~# adduser amavis clamav * Change <code>/etc/clamav/clamd.conf</code> to contain # AllowSupplementaryGroups false AllowSupplementaryGroups true === SpamAssassin === * Edit <code>/etc/default/spamassassin</code> to activate the Spamassassin daemon. Change <code>ENABLED=0</code> to: ENABLED=1 :and, to allow automatic rule updates, change <code>CRON=1</code> CRON=1 * Tell SystemD to keep Spamassassin running. Add these lines to the <code>[Service]</code> section of <code>/lib/systemd/system/spamassassin.service</code>: Restart=on-failure RestartSec=3 Then start Spamassassin: root@server:~# systemctl restart spamassassin.service === Amavis === * Edit <code>/etc/amavis/conf.d/15-content_filter_mode</code> to enable spam filtering @bypass_virus_checks_maps = ( \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re); @bypass_spam_checks_maps = ( \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re); * Edit both <code>/etc/amavis/conf.d/50-user</code> to determine the fate of spam and virus-laden mails $final_virus_destiny = D_PASS; $final_banned_destiny = D_PASS; $final_spam_destiny = D_PASS; $final_bad_header_destiny = D_PASS; :(in other words, don't discard any mail for the moment) * Restart Amavis root@server:~# systemctl restart amavis.service * Once Amavis is working and correctly identifying spam and malware, you can adjust the settings in <code>/etc/amavis/conf.d/50-user</code> to classify more spam, and reject the most spammy. (Rejected messages are quarantined, with a notification going to <code>postmaster@domain.tld</code>.) $enable_dkim_verification = 1; # was disabled to prevent warning $sa_tag_level_deflt = 0.0; # add spam info headers if at, or above that level $sa_tag2_level_deflt = 1.5; # add 'spam detected' headers at that level $sa_kill_level_deflt = 2.5; # triggers spam evasive actions $sa_dsn_cutoff_level = 6.0; # spam level beyond which a DSN is not sent, was 10 $final_bad_header_destiny = D_PASS; # False-positive prone (for spam) $final_virus_destiny = D_DISCARD; # (data not lost, see virus quarantine) $final_banned_destiny = D_REJECT; # D_REJECT when front-end MTA $final_spam_destiny = D_DISCARD; # Where quarantine notifications go $virus_admin = "postmaster\@$mydomain"; # due to D_DISCARD default $spam_admin = "postmaster\@$mydomain"; # due to D_DISCARD default # Where quarantine notifications appear to come from $mailfrom_notify_admin = "contentfilter\@$mydomain"; $mailfrom_notify_recip = $mailfrom_notify_admin; $mailfrom_notify_spamadmin = $mailfrom_notify_admin; $hdrfrom_notify_sender = $mailfrom_notify_admin; $hdrfrom_notify_admin = $mailfrom_notify_admin; Spam will be quarantined and notifications sent to <code>postmaster@domain.tld</code>. The notification will contain lines like: Content type: Spam Internal reference code for the message is 26829-01/ZKBMjq6S2Dsi ... The message has been quarantined as: Z/spam-ZKBMjq6S2Dsi.gz The message WAS NOT relayed to: <someone@domain.tld>: 250 2.7.0 Ok, discarded, id=26829-01 - spam You can release the quarantined message with <code>amavisd-release</code>, like so: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz or send it message to another user by giving their address: root@server:~# amavisd-release Z/spam-ZKBMjq6S2Dsi.gz admin@domain.tld Delete old spam by creating the file <code>/etc/cron.daily/delete_old_spam</code> containing #!/bin/sh find /var/lib/amavis/virusmails/ -type f -mtime +31 -delete and make it executable root@server:~# chmod u+x /etc/cron.daily/delete_old_spam === Postfix === * Now, include Amavis in Postfix's message queue. Modify <code>/etc/postfix/main.cf</code> to use Amavis # Spam and virus filtering content_filter=smtp-amavis:[127.0.0.1]:10024 * Modify <code>/etc/postfix/master.cf</code> to handle the return from Amavis # # Amavis spam and virus filtering # # Sending to Amavis # smtp-amavis unix - - - - 2 smtp smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes -o disable_dns_lookups=yes -o max_use=20 # Returning from Amavis # 127.0.0.1:10025 inet n - - - - smtpd 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_delay_reject=no -o smtpd_client_restrictions=permit_mynetworks,reject -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o smtpd_data_restrictions=reject_unauth_pipelining -o smtpd_end_of_data_restrictions= -o mynetworks=127.0.0.0/8 -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000 -o smtpd_client_connection_count_limit=0 -o smtpd_client_connection_rate_limit=0 -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings * Add the two <code>-o …</code> lines to <code>master.cf</code>, to prevent Spamassassin's reports being checked for being spam. pickup unix n - y 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks * Reload the Postfix configuration: root@server:~# postfix reload ==Enable Dovecot LDA== We now want Postfix to use the Dovecot LDA for local mail delivery and filtering. * Modify <code>/etc/postfix/master.cf</code> to include the line # # dovecot as a local delivery agent # dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} * Modify <code>/etc/postfix/main.cf</code> to include the line ## Local Delivery dovecot_destination_recipient_limit = 1 mailbox_transport = dovecot virtual_transport = dovecot * Reload the Postfix configuration: root@server:~# postfix reload == Block addresses that are known to spammers == The <code>recipient_delimiter</code> option on Postfix is rather neat. Given a <code>recipient_delimiter</code> of '<code>.</code>', messages to <code>user.ext1@domain1.com</code> are handled in an interesting way. If <code>user.ext1@domain1.com</code> is a valid user, the message is put in their mailbox. If <code>user.ext1@domain1.com</code> isn't a valid user, Postfix will drop everything in the username after the delimiter and retry the delivery to <code>user@domain1.com</code>. This means that you can create specialised-use email addresses and have them all come to the same mailbox. It also means you can create 'disposable' email addresses. That also requires some method of disposing of those addresses when the time comes. The disposal is done by checking both the headers of incoming emails the <code>RCPT TO</code> addresses, and discarding or rejecting those that are to a disposable address. * Add the following to <code>/etc/postfix/main.cf</code> # block addresses that are known to spammers header_checks = regexp:/etc/postfix/header_checks smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # Add this directive permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Create the file <code>/etc/postfix/header_checks</code> if /^To:/ /^To:.*(user1\.ext1@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table /^To:.*(user1\.ext2@domain1\.com)/ reject Recipient address rejected: User unknown in virtual mailbox table endif * Create the file <code>/etc/postfix/recipient_checks</code> # Reject messages with these recipient addresses user.banned2@example.com REJECT user.banned@example.com REJECT user.banned3@example.com REJECT user.banned4@example.com REJECT person.banned@example.com REJECT * Reload the configuration root@server:~# postmap /etc/postfix/recipient_checks root@server:~# postfix reload == Configure Postfix as a backup MX server for other domains == Having backup mail servers is always good, so it's only fair to offer the same service to others in exchange. This is how you do it. * In <code>/etc/postfix/main.cf</code>, modify <code>smtpd_recipient_restrictions</code> and add the <code>relay_domains</code> and <code>smtpd_relay_restrictions</code> smtpd_recipient_restrictions = check_recipient_access hash:/etc/postfix/recipient_checks # $mydomain included as a virtual domain below relay_domains = otherdomain1.com otherdomain2.com # Include the relay_recipients to only relay certain addresses at the relayed domains # relay_recipient_maps = hash:/etc/postfix/relay_recipients smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination * Reload the configuration root@server:~# postfix reload However, there is a problem with queue lifetimes. With this setup, messages held in the backup role will only be stored on this server for the standard three days. If you're acting as backup for a privately-run mail server, this may not be long enough (the other site's owner could be away on holiday, for example, and may not get back within three days). Simply increasing the queue lifetime means that undeliverable messages to third parties are not reported back for a long time, which isn't that great. The way to fix is, apparently, is to have two Postfix instances. One instance handles the normal traffic, with a short queue time, while the other handles traffic for the backed-up mail host; the latter Postfix instance has a long queue time and so can keep the mail for a while. I've not attempted to do this yet, as the above settings work OK for me. == Open submission port (587) == Some ISPs prevent access to port 25, the normal mail transfer port. To allow clients on these networks to still connext to this server, we open port 587 for SMTP mail submission. Modify <code>/etc/postfix/master.cf</code> to include: submission inet n - - - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,permit_mx_backup,reject -o milter_macro_daemon_name=ORIGINATING You may also need to modify <code>/etc/postfix/main.cf</code> to include data_directory = /var/lib/postfix smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache tls_random_exchange_name = ${data_directory}/prng_exch And don't forget to open the port in the [[Firewall setup|firewall]] and on the [[Router setup|router]]. ==Authenication and verification== There are a number of ways to authenticate your email servers with others. This should prevent spam messages. ===SPF=== This is a mechanism for the receiving MTA to check that messages it receives came from an MTA that was authorised to send them. It uses DNS records to list the allowed domains. You simply set a DNS record to allow other MTAs to check mail that purports to come from you. To check mail that comes from other domains to your own MTA, you need to install an SPF checking tool and integrate it into Postfix. * Add the following as a <code>TXT</code> record to the DNS entry for each domain you send mail. v=spf1 mx a mx:backup.com ~all :This tells other mail servers that mail servers listed the MX portion of your DNS entry can send email for your domain. It also tells those other servers to accept email sent from your IP number and servers listed in the MX records of backup.com (where backup.com is a domain that acts as a backup MX server for your domain). * Install the SPF checking tool in Postfix root@server:~# aptitude install postfix-policyd-spf-python * Add the SPF checker to <code>/etc/postfix/master.cf</code> policy-spf unix - n n - - spawn user=nobody argv=/usr/bin/policyd-spf * Add the SPF timeout line, and modify <code>smtpd_relay_restrictions</code> in <code>/etc/postfix/main.cf</code> smtpd_relay_restrictions = permit_sasl_authenticated permit_mynetworks reject_unauth_destination check_policy_service unix:private/policy-spf policy-spf_time_limit = 3600s * Modify <code>/etc/postfix-policyd-spf-python/policyd-spf.conf</code> to whitelist any mail servers that act as secondary for this domain. Add this line: Domain_Whitelist = mybackup.domain.com * Test the setup by sending a message from a domain that uses SPF to the server you control. The mail log for your MTA should contain a line like Nov 20 10:26:05 server policyd-spf[18300]: Pass; identity=helo; client-ip=157.56.112.83; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com : and the headers of the message should contain a line like Received-SPF: Pass (sender SPF authorized) identity=helo; client-ip=x.x.x.x; helo=blah.senderdomain.com; envelope-from=user@senderdomain.com; receiver=user@domain.com * Test the setup by sending a message to a domain that uses SPF from your own server. The headers of that message should contain a line like Received-SPF: Pass :followed by a bunch of stuff that shows how the receiving MTA checks SPF records. (Taken from [https://help.ubuntu.com/community/Postfix/SPF Ubuntu community docs for Postfix/SPF]). ===DKIM=== With DKIM, the originating MTA takes a hash of some message headers and the message body, signs it with the server's private key, and embeds the signed hash as an additional header in the message. The public key is published as a DNS TXT record. The receiving MTA recalculates the hash and compares it to the signed version in the message. If they match, the message passes the DKIM authentication. In this example, I have several domains that can send mail. Each has a separate DKIM key. The details are stored in the <code>/etc/opendkim/key_table</code> and <code>/etc/opendkim/signing_table</code> files, rather than in the <code>/etc/opendkim.conf</code> file (the latter is fine if you only have one domain). * Install the packages root@server:~# aptitude install opendkim opendkim-tools * Edit <code>/etc/opendkim.conf</code> # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 Socket inet:8891@localhost PiDFile /var/run/opendkim/opendkim.pid # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector mail KeyTable /etc/opendkim/key_table SigningTable /etc/opendkim/signing_table ExternalIgnoreList /etc/opendkim/trusted_hosts InternalHosts /etc/opendkim/trusted_hosts # Commonly-used options; the commented-out versions show the defaults. #Canonicalization simple #Mode sv #SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1h Background yes UserID opendkim:opendkim Canonicalization relaxed/relaxed DNSTimeout 5 Mode sv SignatureAlgorithm rsa-sha256 SubDomains no #UseASPDiscard no #Version rfc4871 X-Header no * Modify <code>/etc/default/opendkim</code> to include only these non-comment lines. RUNDIR=/var/run/opendkim USER=opendkim GROUP=opendkim PIDFILE=$RUNDIR/$NAME.pid EXTRAAFTER= * Create the directories for the keys (one for each domain) $ root@server:~# mkdir /etc/opendkim $ root@server:~# mkdir /etc/opendkim/example.com $ root@server:~# mkdir /etc/opendkim/example2.com * Create <code>/etc/opendkim/trusted_hosts</code> to show which hosts don't need DKIM authentication. In this case, it's just the hosts my LAN. # local host 127.0.0.1 # local subnets that are trusted and do not need to be verified 192.168.1.0/24 aaaa:bbbb:cccc:dddd/64 * Create the keys for each domain cd /etc/opendkim/example.com opendkim-genkey -s mail -d example.com chown opendkim:opendkim mail.private * Create <code>/etc/opendkim/signing_table</code> example.com mail._domainkey.example.com example1.com mail._domainkey.example1.com example2.com mail._domainkey.example2.com * Place the keys in your domains' DNS records. Create a new subdomain <code>mail._domainkey</code> for each domain (e.g. <code>mail._domainkey.example.com</code>). Create a new TXT record for that new subdomain that contains the core of the relevant part of <code>/etc/opendkim/example.com/mail.txt</code>. If the file contains mail._domainkey IN TXT ( "v=DKIM1; k=rsa; " "p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB" ) ; ----- DKIM key mail for example.com the DNS TXT record for <code>mail._domainkey.example.com</code> should contain v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQU...blah...blah...QIDAQAB * Check the entry is in the domain user@desktop:~$ host -t TXT mail._domainkey.example.com ns1.isp.net Using domain server: Name: ns1.isp.net Address: 100.100.100.1006#53 Aliases: mail._domainkey.example.com descriptive text "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEB...blah...blah...uY/wIDAQAB" * If needed, add the entry to <code>/etc/opendkim/key_table</code>: mail._domainkey.domain.tld domain.tld:mail:/etc/opendkim/domain.tld/mail.private * Modify <code>/etc/postfix/main.cf</code> to include # DKIM Milter milter_protocol = 6 milter_default_action = accept smtpd_milters = inet:localhost:8891 non_smtpd_milters = inet:localhost:8891 :(If you already have milters operating, add the DKIM milter settings to the existing ones milter_default_action = accept milter_protocol = 2 smtpd_milters = inet:localhost:8891,inet:localhost:8892 non_smtpd_milters = inet:localhost:8891,inet:localhost:8892 * Modify <code>/etc/postfix/master.cf</code> to prevent messages being signed when going both to and from Amavis. In the section for Amavis, extend the <code>receive_override_options</code> with <code>no_milters</code>: # Returning from Amavis 127.0.0.1:10025 inet n - - - - smtpd -o content_filter= ... -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters * Restart everything, checking <code>/var/log/mail.log</code> for errors root@server:~# systemctl restart opendkim root@server:~# postfix reload * Test it all works by sending some messages to and from domains with working DKIM services. Outgoing messages should contain a header like DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=mail; t=1448108655; bh=Hciu/taJLWKuURd9G8XyD6VK2YpDwXAHXAfUrZpPfRo=; h=Date:Subject:From:To:From; b=IFbjcnrZcD0SQsL9N2zGaOjhR/j2opmjAmxBgp8+3QejtqPgbNqFjdUOeXkQlh0xo M1nX2VK73v+2D7Og41jZuXQgb8jYn2yUeKLkvJDWNnEOdpmXvRpspuhu2+uqwEk8cj uqkY5jTXUBmn9FKVRRn0Xg8Nzl0jG7HY9pYNPXYY= :If it works, the receiving MTA should also include a header like Authentication-Results: spf=pass (sender IP is 100.100.100.100) smtp.mailfrom=example.com; desitination.com; dkim=pass (signature was verified) header.d=example.com;destination.com; dmarc=bestguesspass action=none header.from=example.com; :Check that the DKIM process works in both directions and for each domain you have. * Once you're confident that DKIM is working, tell the world to reject messages that don't have the DKIM keys. Create a new domain <code>adsp._domainkey.example.com</code> and give it a DNS TXT record containing dkim=all Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://help.ubuntu.com/community/Postfix/DKIM Postfix/DKIM Ubuntu community documentation]. ===DMARC=== See also [https://schipka.com/archives/154 this page] on fixing OpenDmarc configuration. DMARC is a method for MTAs to publish a policy for how their messages should be processed, and allows them to report how well they're handling each others' mail according to those policies. '''DMARC policy for outgoing mail''' You create a DNS TXT record that tells other MTAs about how you want your outgoing messages to be processed. You set up a DMARC milter on your MTA to process incoming messages according to the published DMARC policy. The DMARC milter also logs how you're handling other MTAs' messages and can send reports to them as requested.. * Create a subdomain <code>_dmarc.example.com</code> and a TXT record specifying the DMARC reports you want. <nowiki>v=DMARC1; p=none; rua=mailto:dmarc@example.com; fo=1; adkim=r; aspf=r</nowiki> :(The [http://www.kitterman.com/dmarc/assistant.html? KTS DMARC Record Assistant] will create the records for you. Remove the <code>fo=1;</code> for fewer reports once you're confident things are working.) * Create the dmarc user by modifying <code>/etc/postfix/valiases</code> to include lines dmarc@example.com user@example1.com dmarc@example1.com user@example1.com dmarc@example1.com user@example1.com * Refresh the Postfix config. root@server:~# postmap /etc/postfix/valiases root@server:~# postfix reload You should get messages arriving from other MTAs to say how well you've been doing. '''Testing and logging incoming mail''' Next is to log the details from incoming messages and tell those MTAs how well they've been doing. * Install OpenDMARC root@server:~# aptitude install opendmarc libauthen-sasl-perl python3-psycopg2 :(I need the <code>libauthen</code> package to allow the Opendmarc scripts to connect to my mailserver via SASL, and the <code>psycopg2</code> package to allow the importer to connect to [[PostgreSQL_configuration|Postgres]].) * Update <code>/etc/opendmarc.conf</code> AuthservID mail.domain1.com PidFile /var/run/opendmarc/opendmarc.pid # Change this line! Socket inet:8893@localhost RejectFailures false Syslog true TrustedAuthservIDs mail.domain1.com,mail.domain2.com,mail2.example.com UMask 0002 UserID opendmarc:opendmarc IgnoreHosts /etc/opendmarc/ignore.hosts HistoryFile /var/run/opendmarc/opendmarc.dat #for testing: SoftwareHeader true :where <code>mail.domain1.com,mail.domain2.com,mail2.example.com</code> are the names of mail servers you trust, including your own. I included the names of the other domains this MTA serves. * Create a directory for the OpenDMARC Pid file: root@server:~# mkdir /var/run/opendmarc root@server:~# chown -R opendmarc:opendmarc /var/run/opendmarc/ * Create the directory <code>/etc/opendmarc/</code> and the file <code>/etc/opendmarc/ignore.hosts</code>, containing localhost 192.168.1.0/24 aaaa:bbbb:cccc:dddd::/64 * Update <code>/etc/default/opendmarc</code> to include the line SOCKET="inet:8893@localhost" * Exit <code>/etc/postfix/main.cf</code> to include the extra milter. Add the port specification to these lines: smtpd_milters = inet:localhost:8891,inet:localhost:8893 non_smtpd_milters = inet:localhost:8891,inet:localhost:8893 * Start OpenDMARC and refresh Postfix root@server:~# systemctl restart opendmarc.service root@server:~# postfix reload Send a mail to yourself from a service that uses DMARC, such as GMail. Messages you receive should include headers like: DMARC-Filter: OpenDMARC Filter v1.2.0 mail.domain.org 821E7A4 Authentication-Results: mail.domain.org; dmarc=pass header.from=gmail.com :The first line is the flag from the SoftwareHeader line, and shows that OpenDMARC handled the message. The second line is the DMARC authentication result. If it's all working, you can remove the SoftwareHeader line from <code>/etc/opendmarc.conf</code> and restart OpenDMARC. '''Reporting on incoming mail''' See [[Recording Dmarc feedback]]. {| {{Prettytable}} ! {{Hl3}}|'''Important note on versions''' |- | The instructions below won't generate any messages with Ubuntu 14.04 LTS. The supplied version of OpenDMARC in that version of Ubuntu (1.2.0) has a bug where it fails to record the DMARC rua address, which specifies where the DMARC reports should go. This is fixed in OpenDMARC 1.3.1, which is bundled in Ubuntu 15.04 and later. |} The final part is to log the results of DMARC testing incoming mail, and sending the results back to the originating MTAs. * Edit <code>/usr/share/doc/opendmarc/schema.mysql</code>. Update the default timestamp for <code>lastsent</code> on about line 31 to be CREATE TABLE IF NOT EXISTS requests ( … lastsent TIMESTAMP NOT NULL DEFAULT '1970-01-01 01:01:01', : and uncomment the lines CREATE USER 'opendmarc'@'localhost' IDENTIFIED BY 'changeme'; GRANT ALL ON opendmarc.* to 'opendmarc'@'localhost'; :create a stong password to replace 'changeme'. * Run the script to create the MySQL database to store the reports. root@server:~# mysql -u root -p < /usr/share/doc/opendmarc/schema.sql * Fix the missing database backend selectors in the OpenDMARC scripts. In <code>/usr/sbin/opendmarc-import</code> , <code>/usr/sbin/opendmarc-params</code> , and <code>/usr/sbin/opendmarc-reports</code> , change the DBD lines to use "mysql" not "yes" (you may have to hunt, and it seems already fixed in <code>/usr/sbin/opendmarc-expire</code>), # require DBD::yes; require DBD::mysql; # my $dbscheme = "yes"; my $dbscheme = "mysql"; * Create <code>/etc/cron.daily/opendmarc-report</code> to generate the reports once a day. #!/bin/bash DB_SERVER='localhost' DB_USER='opendmarc' DB_PASS='changeme' DB_NAME='opendmarc' WORK_DIR='/var/run/opendmarc' REPORT_EMAIL='dmarc@domain.com' REPORT_ORG='domain.com' mv ${WORK_DIR}/opendmarc.dat ${WORK_DIR}/opendmarc_import.dat -f touch ${WORK_DIR}/opendmarc.dat chown opendmarc:opendmarc ${WORK_DIR}/opendmarc.dat /usr/sbin/opendmarc-import --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose < ${WORK_DIR}/opendmarc_import.dat /usr/sbin/opendmarc-reports --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose \ --interval=86400 --report-email=${REPORT_EMAIL} --report-org=${REPORT_ORG} /usr/sbin/opendmarc-expire --dbhost=${DB_SERVER} --dbuser=${DB_USER} --dbpasswd=${DB_PASS} --dbname=${DB_NAME} --verbose :where you use the <code>DB_PASS</code> password you generated earlier. * Make the cron script executable: root@server:~# chmod +x /etc/cron.daily/opendmarc-report * Ensure that, for testing at least, you get a copy of every outgoing DMARC report. Modify <code>/etc/postfix/main.cf</code> to include sender_bcc_maps = hash:/etc/postfix/sender_bcc :then create <code>/etc/postfix/sender_bcc</code> to contain the one line /dmarc@domain1.com/ user@domain1.com :and refresh the Postfix configuration root@server:~# postmap /etc/postfix/sender_bcc root@server:~# postfix reload Most of these instructions were taken from [https://www.skelleton.net/2015/03/21/how-to-eliminate-spam-and-protect-your-name-with-dmarc/#dkim How to eliminate spam and protect your name with DMARC], with some from [https://blog.laussat.de/2014/04/28/opendmarc-reporting-tool-fails-not-finding-dbdyes-pm/ Christian Laußat's Linux Blog]. ===DANE=== To add: see documents at http://www.postfix.org/TLS_README.html#client_tls_dane . Will require DNSSEC enabled on the sending domain. === Spamhaus block lists === [https://www.spamhaus.org/ Spamhaus] maintains lists of known spammer address and allows people to use them to block spam. They provide a [https://github.com/spamhaus/spamassassin-dqs plugin for SpamAssassin] that handles the lookups. * Sign up for a [https://www.spamhaus.com/free-trial/sign-up-for-a-free-data-query-service-account/ Data Query Service (DQS) account]. * Find your DQS key on the [https://portal.spamhaus.com/dqs/ customer portal]. * Clone [https://github.com/spamhaus/spamassassin-dqs the plugin source code] to somewhere convenient on your server. * Follow the instructions in the plugin's README file. Note that <code>hbltest.sh</code> is in the <code>3.4.1+/</code> directory. * Insert your key into the config files. root@server:~/spamassassin-dqs/3.4.1+# sed -i -e 's/your_DQS_key/aip7yig6sahg6ehsohn5shco3z/g' sh.cf root@server:~/spamassassin-dqs/3.4.1+# sed -i -e 's/your_DQS_key/aip7yig6sahg6ehsohn5shco3z/g' sh_hbl.cf * Modify <code>sh.pre</code> with the location of SpamAssassin loadplugin Mail::SpamAssassin::Plugin::SH /etc/mail/spamassassin/SH.pm * Copy the plugin into the SpamAssassin directory root@server:~/spamassassin-dqs/3.4.1+# cp SH.pm /etc/mail/spamassassin root@server:~/spamassassin-dqs/3.4.1+# cp sh.cf /etc/mail/spamassassin root@server:~/spamassassin-dqs/3.4.1+# cp sh_scores.cf /etc/mail/spamassassin root@server:~/spamassassin-dqs/3.4.1+# cp sh.pre /etc/mail/spamassassin * Restart SpamAssassin, Amavis, and Postfix root@server:~# systemctl restart spamassassin root@server:~# systemctl restart amavis root@server:~# systemctl restart postfix * Give it about 15 minutes to cache what it needs to, then test it with the [https://blt.spamhaus.com/ Spamhaus blocklist tester]. == See also == Here are a few pages that are useful guides or provide background and context. * [http://wanderingbarque.com/howtos/mailserver/mailserver.html Secure Virtual Mailserver HOWTO]: Postfix + OpenLDAP + Dovecot + Jamm + SASL + SquirrelMail * [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP Virtual Domains, Postfix, Dovecot LDA, and LDAP] * [http://adomas.org/2006/08/postfix-dovecot/ Postfix and Dovecot on Ubuntu Dapper Drake] * [http://www.fatofthelan.com/articles/articles.php?pid=22 How To Install Postfix, Amavis, ClamAV, and Spamassassin on Debian Linux] * [http://wiki.apache.org/spamassassin/SingleUserUnixInstall Single-user SpamAssassin installation] * [http://rimuhosting.com/support/settingupemail.jsp?mta=postfix&t=squirrelmail Setting Up Email: A Postfix/Dovecot HOWTO] * [http://en.gentoo-wiki.com/wiki/Mail_server_using_Postfix_and_Dovecot Gentoo Wiki article on Postfix and Dovecot 2] [[Category:Server setup]] 8f48feed3e16c878f75d654247e657e390cd6dc0 0 1650 2964 2932 2021-02-08T12:47:27Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} The easiest way to install [https://docs.ghost.org/docs/ Ghost] is via [https://hub.docker.com/_/ghost/ the Docker image]. ==Installation== * Create the directory for the blog's files. root@server:~# mkdir -p /var/www/blog.domain.tld/content root@server:~# mkdir -p /var/www/blog.domain.tld/system * Set up the container for the blog. root@server:~# docker run --name name.of.container --restart always \ -p 2368:2368 \ -v /var/www/blog.domain.tld/content:/var/lib/ghost/content \ -e url=http://blog.domain.tld \ ghost * Create the service to start the blog on boot. At some point, something created a symlink for me, pointing from <code>/lib/systemd/system/ghost_blog.domain.tld.service</code> to <code>/var/www/blog.domain.tld/system/files/ghost_blog.domain.tld.service</code>. That file should contain [Unit] Description=Ghost systemd service for blog: blog-domain-tld Documentation=https://docs.ghost.org Requires=docker.service After=docker.service [Service] Environment="NODE_ENV=production" ExecStart=/usr/bin/docker start -a name.of.container ExecStop=/usr/bin/docker stop -t 2 name.of.container Restart=always [Install] WantedBy=multi-user.target * Register the service and restart. root@server:~# systemctl daemon-reload root@server:~# systemctl start ghost_blog.domain.tld.service You should be able to connect to the blog at <code>server.domain.tld:2368</code>. ===Installing additional blogs on the same server=== It's exactly the same procedure as above, but in a different place and with different names. Create a new directory for the content, create a new container (with a differnet name), create a new system control file. The only thing to note is the subsequent containers need their own ports for the host machine, so change the <code>docker run</code> command to have <code>-p 2369:2368</code>, where the first number has changed. Use that port number (<code>2369</code>) in the Apache proxy settings below. === Converting from old, manual installations === I originally set up Ghost manually, using a MySQL database. The container version uses Sqlite3, with the database in the <code>blog.domain.tld/content/data/</code> directory. That needed conversion of the databases. I followed the instructions given for the [https://github.com/dumblob/mysql2sqlite conversion script]. root@server:~# mysqldump --user='backup' --skip-extended-insert --compact --databases blog_domain_tld_prod > blog.sql root@server:~# ./mysql2sqlite dump_blog.sql | sqlite3 ghost.db ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] 72d9d150af4029c833beee170ea4c16cd83d343b 2980 2964 2021-07-04T10:03:57Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== First, install Nginx. To allow Nginx to work on the same box as the [[Web server setup|Apache]], change <code>/etc/nginx/sites-enabled/default</code> to include # listen 80 default_server; # listen [::]:80 default_server; listen 8080 default_server; listen [::]:8080 default_server; I created an additional user for the installation, but I'm not sure that was necessary. Remember to use ''that'' user's password when it asks you for <code>sudo</code> permission! * Install the latest stable [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node], following the standard instructions there. * Install [https://docs.ghost.org/docs/install Ghost command-line interface] * Install [https://docs.ghost.org/docs/cli-install Ghost from the command line], installing the blog in <code>/var/www/blog.domain.tld</code>. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. I skipped SSL setup, instead later manually setting up Nginx to use the [[SSL certificate generation|existing Lets Encrypt certificates]] by editing <code>/etc/nginx/sites-enabled/blog.domain.tld</code> to include the lines: ssl_certificate_key /etc/letsencrypt/live/blog.domain.tld/privkey.pem; ssl_certificate /etc/letsencrypt/live/blog.domain.tld/fullchain.pem; I did try using a Docker-based installation of Ghost, but that didn't work well with passing through requests to generate the images in blog posts. ===Installing additional blogs on the same server=== This is much the same as with installing the first blog, but you must override the defaults for some settings. Before installing a second blog on the same server, manually create the database and the database user: user@server:~$ mysql -u root -p mysql> create database www_other_blog_org_prod; mysql> create user 'ghost_otherblog'@'localhost' identified by 'secretpassword'; grant all privileges on www_other_blog_org_prod.* to 'ghost_otherblog'@'localhost'; Then run the installation as normal: ghostinstall@server:/var/www/otherblog$ ghost install When prompted, give the database details you produced before, and ask Ghost ''not'' to create the <code>ghost</code> database user. Once you've done, take a look in <code>/var/www/otherblog/config.production.json</code> for the port the local Ghost server is running on "url": "<nowiki>https://www.otherblog.org</nowiki>", "server": { "port": 2369, "host": "127.0.0.1" }, Use that port number (<code>2369</code>) in the Apache proxy settings below. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] 95750182f0f43cd4fe9995cf4cddac46d1108307 2982 2980 2021-10-03T13:40:22Z Neil 2 /* Installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== First, install Nginx. To allow Nginx to work on the same box as the [[Web server setup|Apache]], change <code>/etc/nginx/sites-enabled/default</code> to include # listen 80 default_server; # listen [::]:80 default_server; listen 8080 default_server; listen [::]:8080 default_server; I created an additional user for the installation, but I'm not sure that was necessary. Remember to use ''that'' user's password when it asks you for <code>sudo</code> permission! * Install the latest stable [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node], following the standard instructions there. * Install [https://ghost.org/docs/install/ubuntu/ Ghost via the command-line interface], installing the blog in <code>/var/www/blog.domain.tld</code>. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. I skipped SSL setup, instead later manually setting up Nginx to use the [[SSL certificate generation|existing Lets Encrypt certificates]] by editing <code>/etc/nginx/sites-enabled/blog.domain.tld</code> to include the lines: ssl_certificate_key /etc/letsencrypt/live/blog.domain.tld/privkey.pem; ssl_certificate /etc/letsencrypt/live/blog.domain.tld/fullchain.pem; I did try using a Docker-based installation of Ghost, but that didn't work well with passing through requests to generate the images in blog posts. ===Installing additional blogs on the same server=== This is much the same as with installing the first blog, but you must override the defaults for some settings. Before installing a second blog on the same server, manually create the database and the database user: user@server:~$ mysql -u root -p mysql> create database www_other_blog_org_prod; mysql> create user 'ghost_otherblog'@'localhost' identified by 'secretpassword'; grant all privileges on www_other_blog_org_prod.* to 'ghost_otherblog'@'localhost'; Then run the installation as normal: ghostinstall@server:/var/www/otherblog$ ghost install When prompted, give the database details you produced before, and ask Ghost ''not'' to create the <code>ghost</code> database user. Once you've done, take a look in <code>/var/www/otherblog/config.production.json</code> for the port the local Ghost server is running on "url": "<nowiki>https://www.otherblog.org</nowiki>", "server": { "port": 2369, "host": "127.0.0.1" }, Use that port number (<code>2369</code>) in the Apache proxy settings below. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] 306702292199f944e70490ae1456aecd707e6ca1 2985 2982 2022-05-06T09:53:51Z Neil 2 Added notes on Ghosthunter wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== First, install Nginx. To allow Nginx to work on the same box as the [[Web server setup|Apache]], change <code>/etc/nginx/sites-enabled/default</code> to include # listen 80 default_server; # listen [::]:80 default_server; listen 8080 default_server; listen [::]:8080 default_server; I created an additional user for the installation, but I'm not sure that was necessary. Remember to use ''that'' user's password when it asks you for <code>sudo</code> permission! * Install the latest stable [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node], following the standard instructions there. * Install [https://ghost.org/docs/install/ubuntu/ Ghost via the command-line interface], installing the blog in <code>/var/www/blog.domain.tld</code>. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. I skipped SSL setup, instead later manually setting up Nginx to use the [[SSL certificate generation|existing Lets Encrypt certificates]] by editing <code>/etc/nginx/sites-enabled/blog.domain.tld</code> to include the lines: ssl_certificate_key /etc/letsencrypt/live/blog.domain.tld/privkey.pem; ssl_certificate /etc/letsencrypt/live/blog.domain.tld/fullchain.pem; I did try using a Docker-based installation of Ghost, but that didn't work well with passing through requests to generate the images in blog posts. ===Installing additional blogs on the same server=== This is much the same as with installing the first blog, but you must override the defaults for some settings. Before installing a second blog on the same server, manually create the database and the database user: user@server:~$ mysql -u root -p mysql> create database www_other_blog_org_prod; mysql> create user 'ghost_otherblog'@'localhost' identified by 'secretpassword'; grant all privileges on www_other_blog_org_prod.* to 'ghost_otherblog'@'localhost'; Then run the installation as normal: ghostinstall@server:/var/www/otherblog$ ghost install When prompted, give the database details you produced before, and ask Ghost ''not'' to create the <code>ghost</code> database user. Once you've done, take a look in <code>/var/www/otherblog/config.production.json</code> for the port the local Ghost server is running on "url": "<nowiki>https://www.otherblog.org</nowiki>", "server": { "port": 2369, "host": "127.0.0.1" }, Use that port number (<code>2369</code>) in the Apache proxy settings below. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> == Install ghostHunter search == Ghosthunter is a client-side search engine for Ghost blogs. The [https://github.com/jamwise/ghostHunter instructions on the ghosthunter homepage] are pretty good for covering the basics. The [https://www.hauntedthemes.com/how-to-add-search-to-ghost-using-ghosthunter/ notes from Haunted Themes] are a good supplement. However, I had to do a couple of other things. 1. Install the [https://ghost.org/docs/content-api/javascript/#installation Content API] package, so that ghosthunter can see the blog's content. (I think this shouldn't be necessary, but I seemed to require it.) 2. Add the site's host name to </code>/assets/js/main.js</code>, in the <code>content-api-host</code> field in the definition of the <code>config</code> object. 3. Change the <code>prettyPubDate</code> to <code>pubDate</code> in all the places it occurs, including files in the <code>assets/ghosthunter/dist</code> directory. ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] 392b7669eb80ddaa0eca3d4c9ec324dfaf4aa6e9 2994 2985 2022-07-22T10:13:51Z Neil 2 Added startup script wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Discourse setup|Discourse]] |align="center" width="50%"| [[Miscellaneous setup|Miscellaneous]] |} [https://docs.ghost.org/docs/install Ghost] is also a bit opinionated. In particular, it expects to use Nginx as a web server, rather than Apache. ==Installation== First, install Nginx. To allow Nginx to work on the same box as the [[Web server setup|Apache]], change <code>/etc/nginx/sites-enabled/default</code> to include # listen 80 default_server; # listen [::]:80 default_server; listen 8080 default_server; listen [::]:8080 default_server; I created an additional user for the installation, but I'm not sure that was necessary. Remember to use ''that'' user's password when it asks you for <code>sudo</code> permission! * Install the latest stable [https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions Node], following the standard instructions there. * Install [https://ghost.org/docs/install/ubuntu/ Ghost via the command-line interface], installing the blog in <code>/var/www/blog.domain.tld</code>. When asked, specify the URL of the blog as <code>https://blog.domain.tld</code>, noting that it's <code>https</code>, not <code>http</code>. When asked, give <code>server.domain.tld</code> as the hostname of the MySQL server, not the default <code>localhost</code>. I skipped SSL setup, instead later manually setting up Nginx to use the [[SSL certificate generation|existing Lets Encrypt certificates]] by editing <code>/etc/nginx/sites-enabled/blog.domain.tld</code> to include the lines: ssl_certificate_key /etc/letsencrypt/live/blog.domain.tld/privkey.pem; ssl_certificate /etc/letsencrypt/live/blog.domain.tld/fullchain.pem; I did try using a Docker-based installation of Ghost, but that didn't work well with passing through requests to generate the images in blog posts. ===Installing additional blogs on the same server=== This is much the same as with installing the first blog, but you must override the defaults for some settings. Before installing a second blog on the same server, manually create the database and the database user: user@server:~$ mysql -u root -p mysql> create database www_other_blog_org_prod; mysql> create user 'ghost_otherblog'@'localhost' identified by 'secretpassword'; grant all privileges on www_other_blog_org_prod.* to 'ghost_otherblog'@'localhost'; Then run the installation as normal: ghostinstall@server:/var/www/otherblog$ ghost install When prompted, give the database details you produced before, and ask Ghost ''not'' to create the <code>ghost</code> database user. Once you've done, take a look in <code>/var/www/otherblog/config.production.json</code> for the port the local Ghost server is running on "url": "<nowiki>https://www.otherblog.org</nowiki>", "server": { "port": 2369, "host": "127.0.0.1" }, Use that port number (<code>2369</code>) in the Apache proxy settings below. ==Apache proxy== You'll need an [[SSL certificate generation|SSL certificate]] which covers this domain. Create the Apache site file as <code>/etc/apache2/sites-available/blog.domain.tld.conf</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost ServerName blog.domain.tld Redirect permanent / https://blog.domain.tld CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On SSLCertificateKeyFile /etc/letsencrypt/live/blog.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/blog.domain.tld/fullchain.pem ServerName blog.domain.tld RequestHeader set X-Forwarded-Proto "https" ProxyPreserveHost On ProxyRequests Off ProxyPass / http://0.0.0.0:2368/ ProxyPassReverse / http://0.0.0.0:2368/ CustomLog /var/log/apache2/blog.domain.tld.access.log combined ServerSignature off </VirtualHost> :Note the <code>X-Forwarded-Proto</code> request header. Enable the site and reload the Apache configuration: root@server:~# a2ensite blog.domain.tld root@server:~# systemctl reload apache2.service ==Configuration== Visit <code><nowiki>https://blog.domain.tld/ghost</nowiki></code> for final configuration. Install the [https://ghost.org/downloads/ Ghost desktop app] for easier control of the blog. ==Install a theme== Several of the changes below will require changes to the theme, so it's a good idea to get the theme sorted out now. I use a variant on the [https://www.ghostforbeginners.com/willow/ Willow] theme ([https://github.com/NeilNjae/ghost-theme-willow my fork on Github]). * Clone the theme, from the Github source, to your desktop machine. * In the theme directory, install Grunt and get it watching the file change. (This will re-create the <code>assets/css/styles.css</code> when there are changes in the files its derived from. user@desktop:~/blog/theme$ npm install user@desktop:~/blog/theme$ grunt * Alter <code>package.json</code> to reflect the changes you're about to make. * Alter <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/partials/sidebar.hbs partials/sidebar.hbs]</code> to adjust the sidebar links. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) * Adjust <code>source/sass/components/_post-view.scss</code> to allow floating images. In the "Post view" section near the top of the file, add these three lines: img[src$="#left"] { max-width:30%; float: left; } img[src$="#right"] { max-width:30%; float: right; } img[src$="#full"] { max-width:none;width:100vw } :You should see the Grunt watcher rebuild the style sheet now. * Move to the directory above and zip the theme: user@desktop:~/blog$ zip -x '*/node_modules/*' '*/.git/*' -r ghost-theme-willow.zip theme * Use the Ghost desktop app to upload the theme. If you change styles, you'll also need to restart the Ghost server. Changes to <code>.hbs</code> files take effect immediately. * Commit your changes into Git, as you would for any other project. * Add some images to your blog. ** ''Publication icon'' is the favicon, and should be 60×60 pixels. ** ''Publication logo'' is a small symbol used in the sidebar header, about 135×135 pixels. ** ''Publication cover'' is a large background image used in the sidebar. Mine's a 550×250 pixel Jpeg. You can now add floating images with by appending <code>#left</code>, <code>#right</code>, or <code>#full</code> to the image URL when creating a post. ==Install Disqus== The [https://help.ghost.org/article/15-disqus standard instructions] are good. Remember to: * Note the Disqus site shortname. * Add <code>blog.domain.tld</code> to Disqus as a trusted domain * Disable the "tracking" and "affiliated sites" in [https://worknjae.disqus.com/admin/settings/advanced/ Disqus advanced settings] * Update the theme's <code>[https://github.com/NeilNjae/ghost-theme-willow/blob/master/post.hbs post.hbs]</code> file to include the Disqus code. (Look on Github, as Mediawiki doesn't like paired curly brackets in markup.) :Note that you should use you own site's Disqus shortname. * Again, zip and upload the modified theme. ==Install MathJax and Prism== MathJax renders Latex formulae. Prism does syntax highlighting. It can all be done by code injection. * In the ''Code Injection'' tab of the desktop app, add these lines to the blog header: <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/themes/prism.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.css" rel="stylesheet"/> <link href="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.css" rel="stylesheet"/> * and add these lines to the blog footer: <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/prism.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/toolbar/prism-toolbar.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/show-language/prism-show-language.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/plugins/line-numbers/prism-line-numbers.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-clike.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-javascript.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-latex.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-markdown.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-haskell.min.js"></script> <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/prism/1.12.2/components/prism-python.min.js"></script> <script type="text/javascript" async src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-MML-AM_CHTML"> MathJax.Hub.Config({ tex2jax: { inlineMath: [["$", "$"], ["\\(", "\\)"]], processEscapes: true } }); </script> Add additional Prism lines to the footer for additional languages. ===Prism markup examples=== When adding code blocks to posts, use this formatting: ```python def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines ``` If you want line numbers: <nowiki><pre><code class="language-python line-numbers">def tpack(text, width=100): """Pack a list of words into lines, so long as each line (including intervening spaces) is no longer than _width_""" lines = [text[0]] for word in text[1:]: if len(lines[-1]) + 1 + len(word) <= width: lines[-1] += (' ' + word) else: lines += [word] return lines </code></pre></nowiki> ===LaTeX markup examples=== For inline equations, use <code>\\(</code> and <code>\\)</code> to delimit the equation, so And then some more text \\(\alpha \in \Gamma\\) to finish will render as : And then some more text <math>\alpha \in \Gamma</math> to finish For block equations, use either <code>$$</code> or <code>\\[</code> / <code>\\]</code> to delimit the blocks, so $$\prod_{\mathfrak{p} \in \Omega}\(\frac{\alpha,-1}{\mathfrak{p}})=1$$ and \\[(X,\beta) \oplus (X,-\beta) \text{ is split }.\\] will render as: <math>\prod_{\mathfrak{p} \in \Omega} \left ( \frac{\alpha, -1}{\mathfrak{p}} \right ) = 1</math> and <math>(X,\beta) \oplus (X,-\beta) \text{ is split }.</math> == Install ghostHunter search == Ghosthunter is a client-side search engine for Ghost blogs. The [https://github.com/jamwise/ghostHunter instructions on the ghosthunter homepage] are pretty good for covering the basics. The [https://www.hauntedthemes.com/how-to-add-search-to-ghost-using-ghosthunter/ notes from Haunted Themes] are a good supplement. However, I had to do a couple of other things. 1. Install the [https://ghost.org/docs/content-api/javascript/#installation Content API] package, so that ghosthunter can see the blog's content. (I think this shouldn't be necessary, but I seemed to require it.) 2. Add the site's host name to </code>/assets/js/main.js</code>, in the <code>content-api-host</code> field in the definition of the <code>config</code> object. 3. Change the <code>prettyPubDate</code> to <code>pubDate</code> in all the places it occurs, including files in the <code>assets/ghosthunter/dist</code> directory. == Startup scripts == The blog engine needs to start up when the system boots. Create the file <code>/lib/systemd/system/ghost_blog_domain_tld.service</code> containing [Unit] Description=Ghost systemd service for blog: blog.domain.tld Documentation=https://ghost.org/docs/ After=network-online.target remote-fs.target nss-lookup.target [Service] Type=simple WorkingDirectory=/var/www/blog.domain.tld User=999 Environment="NODE_ENV=production" ExecStart=/usr/bin/node /usr/bin/ghost run Restart=always [Install] WantedBy=multi-user.target Ensure that SystemD knows about the new service: root@server:~# systemctl daemon-reload Start the service: root@server:~#:systemctl start ghost_blog_domain_tld.service That should be enough for the blog engine to start up on each boot. ==See also== * [http://blog.benoitblanchon.fr/install-ghost-and-apache-ubuntu/ notes on installing Ghost with Apache] * [https://mrvautin.com/how-to-setup-https-on-your-ghost-blog-without-redirect-loop/ mrvautin's post on avoiding redirect loops]. * [https://www.ghostforbeginners.com/how-to-enable-comments-on-a-ghost-blog/ Adding comments to Ghost] * [https://medium.com/zurassic/how-to-add-comments-disqus-to-ghost-b81b75d28fda More on adding comments to Ghost] * [https://themes.ghost.org/docs/ Ghost theme documentation] * [https://themes.ghost.org/docs/navigation Ghost navigation documentation] * [https://bell0bytes.eu/syntax-latex/ Adding Prism and Latex] * Other themes I like: [https://www.ghostforbeginners.com/coder/ Coder], [https://www.ghostforbeginners.com/promenade/ Promenade], and [https://www.ghostforbeginners.com/ghoststa/ Ghoststa] [[Category:Server setup]] de573dfece1998da704eb79d10c09a361f0d0183 0 1535 2967 2935 2021-02-11T10:27:26Z Neil 2 /* Installation */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Mediawiki farm setup|Mediawiki farm]] |align="center" width="50%"| [[CUPS server setup|Print server]] |} == Installation == This one's easy, once the [[Web server setup#Secure_HTTP|HTTPS web server]] is set up. I'm using [https://roundcube.net/ Roundcube] as a webmail interface. * Install the packages: root@server:~# aptitude install roundcube roundcube-pgsql roundcube-plugins :(Install other <code>aspell-</code> dictionaries as you wish at the same time.) * Follow the defaults in the questions asked during installation. However, these questions don't address all the configuration options. Therefore, after installation, reconfigure the packages: root@server:~# sudo dpkg-reconfigure roundcube-core * Roundcube wants to make itself available as <code>domain.tld/roundcube</code> on all sites. The file <code>/etc/apache2/conf-available/roundcube.conf</code> does this with the <code>Alias /roundcube /var/lib/roundcube</code> directive, which is initially commented out. Keep it commented out, and copy the contents of that file into the <code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>, uncommenting the <code>Alias</code> line. * Disable the Roundcube configuration, enable the webmail site, and reload Apache: root@server:~# a2disconf roundcube root@server:~# a2ensite webmail.domain.tld.conf root@server:~# systemctl reload apache2.service * Visit <code><nowiki>https://webmail.domain.tld</nowiki></code> and you should be able to log on. * If need be, add this line to <code>/etc/roundcube/config.inc.php</code> to specify the outgoing SMTP server. $config['smtp_server'] = 'tls://mail.%t'; ==Installing plugins== Where possible, use the [https://plugins.roundcube.net/ Roundcube plugins installer], working from the <code>/var/lib/roundcube</code> directory. ===Mobile skin=== Roundcube doesn't work well on mobile devices. The [https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile Melanie2 skin] seems to be a good alternative. The [https://roundcubeinbox.wordpress.com/2016/04/26/roundcube-for-mobile-devices/ instructions at the Roundcube inbox] are slightly unclear with paths, so here's what I did. root@server:~# cd /var/lib/roundcube root@server:/var/lib/roundcube# git clone <nowiki>https://github.com/messagerie-melanie2/Roundcube-Skin-Melanie2-Larry-Mobile.git</nowiki> skins/melanie2_larry_mobile root@server:/var/lib/roundcube# php composer.phar update root@server:/var/lib/roundcube# php composer.phar require melanie2/mobile:dev-master :remembering to say "no" to activating the <code>jquery_mobile</code> plugin, but "yes" to activating the <code>mobile</code> plugin. ===Enigma=== The Enigma plugin comes as part of the Roundcube installation. To enable it, add <code>enigma</code> to the list of enabled plugins in <code>/etc/roundcube/config.inc.php</code>. (Note that this doesn't seem to work at the moment.) ==<code>/etc/apache2/sites-available/webmail.domain.tld.conf</code>== <VirtualHost *:443> ServerAdmin webmaster@localhost SSLEngine On # SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/fullchain.pem DocumentRoot /var/lib/roundcube ServerName webmail.domain.tld <Directory /> Options +FollowSymLinks -Indexes AllowOverride None </Directory> <Directory /var/lib/roundcube/> Options +FollowSymLinks # This is needed to parse /var/lib/roundcube/.htaccess. See its # content before setting AllowOverride to None. AllowOverride All <IfVersion >= 2.3> Require all granted </IfVersion> <IfVersion < 2.3> Order allow,deny Allow from all </IfVersion> </Directory> # Protecting basic directories: <Directory /var/lib/roundcube/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/lib/roundcube/temp> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> <Directory /var/lib/roundcube/logs> Options -FollowSymLinks AllowOverride None <IfVersion >= 2.3> Require all denied </IfVersion> <IfVersion < 2.3> Order allow,deny Deny from all </IfVersion> </Directory> ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/webmail.domain.tld.access.log combined ServerSignature Off </VirtualHost> == See also == * [https://github.com/roundcube/roundcubemail/wiki Roundcube documentation] * [https://help.ubuntu.com/community/Roundcube Installing Roundcube on Ubuntu] [[Category:Server setup]] 86976234b35ad238b16cdac47130f00ba8069443 0 1645 2968 2884 2021-02-11T17:27:33Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[NTP client setup|NTP]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://certbot.eff.org/instructions install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== This is a bit of chicken-and-egg. First, you need to create the non-HTTPS virtual sites for all the domains you want to serve. You can then use the Apache plugin to the LetsEncrypt client to fetch and install the certificates. Then, you can go back and fix the various certificate settings in the virtual host conf files. If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# certbot --apache -d domain.tld -d www.domain.tld root@server:# certbot --apache -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# certbot --apache -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. Certificates will expire in about 90 days, but Certbot will automatically renew them before that. You can test renewal with this command: root@ogedei:~# certbot renew --dry-run and check the SystemD timer with root@ogedei:~# systemctl list-timers ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--cert-name</code> option and list ''all'' the domains for the certificate: root@server:~# certbot certonly --cert-name domain.tld -d domain.tld,www.domain.tld,other.domain.tld root@server:~# systemctl reload apache2.service Domains not listed in the <code>certbot --cert-name</code> command will be removed from the certificate. Note that you need to include the original <code>domain.tld</code> certificate name in the certificate expansion command. Luckily, <code>certbot</code> asks you to confirm changes before you make them. ==Check the contents of a certificate== If you want to see what certificates you have, use `certbot`: root@server:~# certbot certificates If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] d3be2d6896d5572e1d5f1ad93638f6f6be5f67f8 2974 2968 2021-04-13T14:04:16Z Neil 2 Removed NTP wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[DNS server setup|DNS]] |align="center" width="50%"| [[Dovecot server setup|MDA (Dovecot)]] |} ==Install the Let's Encrypt client== [https://letsencrypt.org/ Let's Encrypt] offer free SSL certificates for use in HTTPS and similar services (such as IMAP and SMTP over TLS). The first thing to do is [https://certbot.eff.org/instructions install the Let's Encrypt client] '''on the server'''. ==Get certificates from Lets Encrypt== This is a bit of chicken-and-egg. First, you need to create the non-HTTPS virtual sites for all the domains you want to serve. You can then use the Apache plugin to the LetsEncrypt client to fetch and install the certificates. Then, you can go back and fix the various certificate settings in the virtual host conf files. If you have any existing web server running, turn it off for the first creation of certificates. Get the certificates from: root@server:# certbot --apache -d domain.tld -d www.domain.tld root@server:# certbot --apache -d other.domain.tld -d www.other.domain.tld -d other1.domain.tld -d www.other1.domain.tld root@server:# certbot --apache -d webmail.domain.tld -d mail.domain.tld -d imap.domain.tld Each command will create a single certificate, with aliases to cover all the domains specified in each command. That will create certificates in <code>/etc/letsencrypt/live/domain.tld</code> and so on, one directory for each certificate obtained. Note that I've also got a certificate for my mail servers, used for IMAP and SMTP over TLS. If you use the <code>--apache</code> option in the command, it will automatically include the certificate use settings in the website config files. If you do that, note that every domain listed in all the requests must be explicitly listed as a <code>ServerName</code> or <code>ServerAlias</code> in an Apache conf file, and that no Apache conf file can contain more than one <code>VirtualHost</code> section on port 443. Certificates will expire in about 90 days, but Certbot will automatically renew them before that. You can test renewal with this command: root@ogedei:~# certbot renew --dry-run and check the SystemD timer with root@ogedei:~# systemctl list-timers ==Extend the domains of a certificate== If you want to add additional domains to an existing certificate, you need the <code>--cert-name</code> option and list ''all'' the domains for the certificate: root@server:~# certbot certonly --cert-name domain.tld -d domain.tld,www.domain.tld,other.domain.tld root@server:~# systemctl reload apache2.service Domains not listed in the <code>certbot --cert-name</code> command will be removed from the certificate. Note that you need to include the original <code>domain.tld</code> certificate name in the certificate expansion command. Luckily, <code>certbot</code> asks you to confirm changes before you make them. ==Check the contents of a certificate== If you want to see what certificates you have, use `certbot`: root@server:~# certbot certificates If you want to check the contents of a certificate, use this command: root@server:~# openssl x509 -in /etc/letsencrypt/live/domain.tld/cert.pem -text Areas of interest are likely to be the <code>Validity</code> section, which contains the date range for the certificate being valid, and the <code>X509v3 Subject Alternative Name</code> section, which lists the domains for which this certificate is valid. ==See also== Read the old instructions for [[Self-signed SSL certificate generation]]. [[Category:Server setup]] f311393e14eaaa2636ea469f9fd44697b6ab2ead 0 1631 2970 2920 2021-04-08T09:59:11Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client On the desktop, install <code>pgadmin4</code> as well root@server:~# curl https://www.pgadmin.org/static/packages_pgadmin_org.pub > packages_pgadmin_org.pub root@server:~# gpg --no-default-keyring --keyring ./temp-keyring.gpg --import packages_pgadmin_org.pub root@server:~# gpg --no-default-keyring --keyring ./temp-keyring.gpg --export --output packages_pgadmin_org.gpg root@server:~# cp packages_pgadmin_org.gpg /usr/local/share/keyrings/ root@server:~# echo "deb [signed-by=/usr/local/share/keyrings/packages_pgadmin_org.gpg] https://ftp.postgresql.org/pub/pgadmin/pgadmin4/apt/$(lsb_release -cs) pgadmin4 main" root@server:~# aptitude update && aptitude install pgadmin4 (Following [https://askubuntu.com/questions/1286545/what-commands-exactly-should-replace-the-deprecated-apt-key this post] for GPG keys.) == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf</code> and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/9.1/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == Updating == It's perfectly possible to have several instances of PostreSQL running at different versions, such as 9.5 and 10.1. To migrate the data and remove the old version, [https://stackoverflow.com/questions/46687645/upgrade-postgresql-from-9-6-to-10-0-on-ubuntu-16-10 follow these steps]: * Use `pg_lsclusters` to see what's running. root@desktop:~# pg_lsclusters Ver Cluster Port Status Owner Data directory Log file 9.5 main 5432 online postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log 10 main 5433 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log * Check what's in what cluster root@server::# su - postgres postgres@server:~$ psql --cluster 9.5/main postgres=# select datname from pg_database; datname ----------- template1 template0 postgres ... ... (5 rows) postgres=# \q :(where `...` indicates some more databases in the cluster) postgres@server:~$ psql --cluster 10/main psql (10.5 (Ubuntu 10.5-0ubuntu0.18.04)) postgres=# select datname from pg_database; datname ----------- postgres template1 template0 (3 rows) : this indicates that there's nothing currently in the version 10 cluster. * Stop Posgres root@server:~# systemctl stop postgresql * Destroy the existing empty cluster in the version 10 cluster root@server:~# pg_dropcluster --stop 10 main * Move the existing data to version 10 root@server:~# pg_upgradecluster -m upgrade 9.5 main * Drop the 9.5 cluster root@server:~# pg_dropcluster --stop 9.5 main * Use `aptitude` to purge all the postgresql-9.5 packages. * Restart Postgres root@server:~# systemctl start postgresql == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] 9729aa05bff6cc931bcea078f44cbf11b13928fb 2971 2970 2021-04-08T11:19:22Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client On the desktop, install <code>pgadmin4</code> as well root@server:~# curl https://www.pgadmin.org/static/packages_pgadmin_org.pub > packages_pgadmin_org.pub root@server:~# gpg --no-default-keyring --keyring ./temp-keyring.gpg --import packages_pgadmin_org.pub root@server:~# gpg --no-default-keyring --keyring ./temp-keyring.gpg --export --output packages_pgadmin_org.gpg root@server:~# cp packages_pgadmin_org.gpg /usr/share/keyrings/ root@server:~# echo "deb [signed-by=/usr/share/keyrings/packages_pgadmin_org.gpg] https://ftp.postgresql.org/pub/pgadmin/pgadmin4/apt/$(lsb_release -cs) pgadmin4 main" root@server:~# aptitude update && aptitude install pgadmin4 (Following [https://askubuntu.com/questions/1286545/what-commands-exactly-should-replace-the-deprecated-apt-key this post] for GPG keys.) == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf</code> and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 Edit <code>/etc/postgresql/9.1/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == Updating == It's perfectly possible to have several instances of PostreSQL running at different versions, such as 9.5 and 10.1. To migrate the data and remove the old version, [https://stackoverflow.com/questions/46687645/upgrade-postgresql-from-9-6-to-10-0-on-ubuntu-16-10 follow these steps]: * Use `pg_lsclusters` to see what's running. root@desktop:~# pg_lsclusters Ver Cluster Port Status Owner Data directory Log file 9.5 main 5432 online postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log 10 main 5433 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log * Check what's in what cluster root@server::# su - postgres postgres@server:~$ psql --cluster 9.5/main postgres=# select datname from pg_database; datname ----------- template1 template0 postgres ... ... (5 rows) postgres=# \q :(where `...` indicates some more databases in the cluster) postgres@server:~$ psql --cluster 10/main psql (10.5 (Ubuntu 10.5-0ubuntu0.18.04)) postgres=# select datname from pg_database; datname ----------- postgres template1 template0 (3 rows) : this indicates that there's nothing currently in the version 10 cluster. * Stop Posgres root@server:~# systemctl stop postgresql * Destroy the existing empty cluster in the version 10 cluster root@server:~# pg_dropcluster --stop 10 main * Move the existing data to version 10 root@server:~# pg_upgradecluster -m upgrade 9.5 main * Drop the 9.5 cluster root@server:~# pg_dropcluster --stop 9.5 main * Use `aptitude` to purge all the postgresql-9.5 packages. * Restart Postgres root@server:~# systemctl start postgresql == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] c2cad76c1c0436308739d32be65a19ed99478d41 3000 2971 2022-07-23T16:29:51Z Neil 2 /* Remote access */ ipv6 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[MySQL configuration|MySQL config]] |align="center" width="50%"| [[Web server setup|Web server]] |} If Posgtres isn't already installed, install it: root@server:~# aptitude install postgresql postgresql-client On the desktop, install <code>pgadmin4</code> as well root@server:~# curl https://www.pgadmin.org/static/packages_pgadmin_org.pub > packages_pgadmin_org.pub root@server:~# gpg --no-default-keyring --keyring ./temp-keyring.gpg --import packages_pgadmin_org.pub root@server:~# gpg --no-default-keyring --keyring ./temp-keyring.gpg --export --output packages_pgadmin_org.gpg root@server:~# cp packages_pgadmin_org.gpg /usr/share/keyrings/ root@server:~# echo "deb [signed-by=/usr/share/keyrings/packages_pgadmin_org.gpg] https://ftp.postgresql.org/pub/pgadmin/pgadmin4/apt/$(lsb_release -cs) pgadmin4 main" root@server:~# aptitude update && aptitude install pgadmin4 (Following [https://askubuntu.com/questions/1286545/what-commands-exactly-should-replace-the-deprecated-apt-key this post] for GPG keys.) == Passwords == * Connect to the Postres server: root@server:~# sudo -u postgres psql postgres * Set a password for the "postgres" database role using the command: postgres=# \password postgres :Ctrl-D to exit == Remote access == Edit <code>/etc/postgresql/9.1/main/pg_hba.conf</code> and modify the default configuration which is very protective. For example, if you want postgres to manage its own users (not linked with system users), you will add the following lines: # TYPE DATABASE USER IP-ADDRESS IP-MASK METHOD # Allow connections from across the LAN host all all 192.168.1.0 255.255.255.0 md5 # Allow any connection from this machine local all all md5 host all all 127.0.0.1/32 md5 # IPv6 local connections: host all all ::1/128 md5 host all all aaaa:bbbb:cccc:dddd::/64 md5 Edit <code>/etc/postgresql/9.1/main/postgresql.conf</code> to include listen_addresses = '192.168.1.251,localhost' Note that the [[Firewall setup|firewall]] is configured to only allow connections to PostgreSQL from machines on the LAN. == Updating == It's perfectly possible to have several instances of PostreSQL running at different versions, such as 9.5 and 10.1. To migrate the data and remove the old version, [https://stackoverflow.com/questions/46687645/upgrade-postgresql-from-9-6-to-10-0-on-ubuntu-16-10 follow these steps]: * Use `pg_lsclusters` to see what's running. root@desktop:~# pg_lsclusters Ver Cluster Port Status Owner Data directory Log file 9.5 main 5432 online postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log 10 main 5433 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log * Check what's in what cluster root@server::# su - postgres postgres@server:~$ psql --cluster 9.5/main postgres=# select datname from pg_database; datname ----------- template1 template0 postgres ... ... (5 rows) postgres=# \q :(where `...` indicates some more databases in the cluster) postgres@server:~$ psql --cluster 10/main psql (10.5 (Ubuntu 10.5-0ubuntu0.18.04)) postgres=# select datname from pg_database; datname ----------- postgres template1 template0 (3 rows) : this indicates that there's nothing currently in the version 10 cluster. * Stop Posgres root@server:~# systemctl stop postgresql * Destroy the existing empty cluster in the version 10 cluster root@server:~# pg_dropcluster --stop 10 main * Move the existing data to version 10 root@server:~# pg_upgradecluster -m upgrade 9.5 main * Drop the 9.5 cluster root@server:~# pg_dropcluster --stop 9.5 main * Use `aptitude` to purge all the postgresql-9.5 packages. * Restart Postgres root@server:~# systemctl start postgresql == See also == Here are a few pages that are useful guides or provide background and context. * [https://help.ubuntu.com/community/PostgreSQL Ubuntu PostgreSQL documentation] * [http://www.postgresql.org/docs/9.1/static/index.html PostgreSQL 9.1 documentation] [[Category:Server setup]] 012ff865e64dcd5de7021c4bb37f4c33fe3bbf49 0 1570 2972 2799 2021-04-13T14:02:41Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[DNS server setup|DNS]] |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |} No longer needed. The default systemd-timesyncd feature is sufficient for normal use. 5a900334e7c312ba13ef7938338e73e57e19c33a 0 1544 2973 2830 2021-04-13T14:04:13Z Neil 2 Removed NTP wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# aptitude install bind9 :This will install the configuration files in <code>/etc/bind</code> * Alter the <code>/etc/bind/named.conf.options</code> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; dnssec-validation auto; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; :if you have problems with your forwarders not implementing DNSSEC (<code>error (insecurity proof failed) resolving './NS/IN'</code> appearing in <code>/var/log/syslog</code> and intermittent DNS service), you may want to replace the dnssec lines with: // dnssec-validation auto; dnssec-enable no; dnssec-validation no; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <code>/etc/bind/named.conf.local</code> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; also-notify { 192.168.1.251; }; allow-transfer { 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { 192.168.1.252; }; allow-notify { 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <code>/var/cache/bind</code> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <code>/etc/bind/</code>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <code>/etc/bind/db.domain.tld</code> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2008051601 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (NS) ; and mail exchange (MX) NS server.domain.tld. NS desktop.domain.tld. MX 10 server.domain.tld. ; define some aliases ns1 CNAME server ns2 CNAME desktop mail CNAME server imap CNAME server smtp CNAME server www CNAME server scripts CNAME server printer CNAME desktop printers CNAME desktop webmail CNAME server ; define other servers localhost A 127.0.0.1 desktop A 192.168.1.251 server A 192.168.1.252 router A 192.168.1.1 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone file, <code>/etc/bind/db.1.168.192</code> is (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. :(Note the full stops after all the domain names.) * Update <code>/etc/network/interfaces</code> to tell <code>dnsmasq</code> about the nameservers # The primary network interface auto enp0s7 iface enp0s7 inet static address 192.168.1.251 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * Restart bind: root@server:~# systemctl restart bind9.service :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. For DD-WRT, the settings to use are: * '''Router IP''' ** ''Local DNS:'' 192.168.0.252 * '''Network Address Server Settings (DHCP)''' ** ''Static DNS 1:'' 192.168.0.252 ** ''Static DNS 2:'' 192.168.0.251 ** ''Static DNS 3:'' 0.0.0.0 ** ''WINS:'' 0.0.0.0 ** ''Use DNSMasq for DHCP:'' OFF ** ''Use DNSMasq for DNS:'' OFF ** ''DHCP-Authoritative:'' OFF == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <code>/usr/share/doc/bind9/README.Debian.gz</code> file on your machine's local disk has some good pointers. [[Category:Server setup]] 624e078d45123ca8f72ca71fb0b72fbce103934b 3001 2973 2022-07-24T14:58:08Z Neil 2 Added ipv6 details wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Protect SSH]] |align="center" width="50%"| [[SSL certificate generation|SSL certificates]] |} Name servers are necessary so that the various client machines on the LAN can find the services offered by the servers. Setting up the Domain Name servers is a three-stage process. The first stage is to get the name servers running. The second stage is about making them secure by placing them in a chroot jail. The final stage is to make all the other machines on the LAN use these new nameservers, so that they can see the various servers on the LAN. == Getting the servers running == First, create the master nameserver on Server: * Install bind (make sure it's BIND9, not BIND (which is BIND8)): root@server:~# aptitude install bind9 :This will install the configuration files in <code>/etc/bind</code> * Alter the <code>/etc/bind/named.conf.options</code> file to mention my ISP's nameservers options { directory "/var/cache/bind"; forwarders { 1.2.3.4; 1.2.3.5; }; dnssec-validation auto; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; allow-recursion { localnets; }; }; :if you have problems with your forwarders not implementing DNSSEC (<code>error (insecurity proof failed) resolving './NS/IN'</code> appearing in <code>/var/log/syslog</code> and intermittent DNS service), you may want to replace the dnssec lines with: // dnssec-validation auto; dnssec-enable no; dnssec-validation no; * Tell BIND about all the domains I want machines on my LAN to know about by mentioning them in <code>/etc/bind/named.conf.local</code> This is the file for the master server: zone "domain.tld" { type master; file "/etc/bind/db.domain.tld"; notify-source-v6 aaaa:bbbb:cccc:dddd::53; also-notify { aaaa:bbbb:cccc:dddd::54; 192.168.1.251; }; allow-transfer { aaaa:bbbb:cccc:dddd::54; 192.168.1.251; }; }; zone "other-domain.org" { type master; file "/etc/bind/db.other-domain.org"; notify-source-v6 aaaa:bbbb:cccc:dddd::53; also-notify { aaaa:bbbb:cccc:dddd::54; 192.168.1.251; }; allow-transfer { aaaa:bbbb:cccc:dddd::54; 192.168.1.251; }; }; zone "1.168.192.in-addr.arpa" { type master; file "/etc/bind/db.1.168.192"; notify-source-v6 aaaa:bbbb:cccc:dddd::53; also-notify { aaaa:bbbb:cccc:dddd::54; 192.168.1.251; }; allow-transfer { aaaa:bbbb:cccc:dddd::54; 192.168.1.251; }; }; :(note the absolute file names for the zone files) :and here is the file for the slave zone "domain.tld" { type slave; file "db.domain.tld"; masters { aaaa:bbbb:cccc:dddd::53; 192.168.1.252; }; allow-notify { aaaa:bbbb:cccc:dddd::53; 192.168.1.252; }; }; zone "other-domain.org" { type slave; file "db.other-domain.org"; masters { aaaa:bbbb:cccc:dddd::53; 192.168.1.252; }; allow-notify { aaaa:bbbb:cccc:dddd::53; 192.168.1.252; }; }; zone "1.168.192.in-addr.arpa" { type slave; file "db.1.168.192"; masters { aaaa:bbbb:cccc:dddd::53; 192.168.1.252; }; allow-notify { aaaa:bbbb:cccc:dddd::53; 192.168.1.252; }; }; zone "d.d.d.d.c.c.c.c.b.b.b.b.a.a.a.a.ip6.arpa" { type slave; file "db.aaaa.bbbb.cccc.dddd"; masters { aaaa:bbbb:cccc:dddd::53; 192.168.1.252; }; allow-notify { aaaa:bbbb:cccc:dddd::53; 192.168.1.252; }; }; :Note the relative paths of the zone files. This is because BIND's default working directory is <code>/var/cache/bind</code> and the slave's zone files will be stored there. The AppArmour security module prevents BIND itself from saving files into <code>/etc/bind/</code>, so slave zone files can't be stored in that directory. * Now, create the zone files themselves, such as <code>/etc/bind/db.domain.tld</code> . Only do this on the master server, as the zone files will be automatically transferred to the slave server. ; ; BIND data file for the domain.tld domain ; $ORIGIN domain.tld. $TTL 60 @ IN SOA server.domain.tld. root.domain.tld. ( 2022072401 ; Serial 1d ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; server.domain.tld serves this domain as nameserver (IN NS) ; and mail exchange (IN MX) IN NS ns1.domain.tld. IN NS ns2.domain.tld. IN MX 10 mail.domain.tld. ; define the base domain name domain.tld. IN AAAA aaaa:bbbb:cccc:dddd::2 domain.tld. IN A 192.168.1.252 ; define other servers localhost IN AAAA ::1 localhost IN A 127.0.0.1 ;;;;;;;;;;;; ; machines desktop IN AAAA aaaa:bbbb:cccc:dddd::1 IN A 192.168.1.251 server IN AAAA aaaa:bbbb:cccc:dddd::2 IN A 192.168.1.252 router IN AAAA 2a02:390:62aa::1 IN A 10.191.106.1 ; Name servers ns1 IN AAAA aaaa:bbbb:cccc:dddd::53 IN A 192.168.1.252 ns2 IN AAAA aaaa:bbbb:cccc:dddd::54 IN A 192.168.1.251 ; ns2 IN CNAME desktop ; Web servers www IN AAAA aaaa:bbbb:cccc:dddd::443 IN A 192.168.1.252 webmail IN CNAME www ; Mail servers mail IN AAAA aaaa:bbbb:cccc:dddd::25 IN A 192.168.1.252 imap IN AAAA aaaa:bbbb:cccc:dddd::143 IN A 192.168.1.252 smtp IN CNAME mail ; Git git IN AAAA aaaa:bbbb:cccc:dddd::9418 IN A 192.168.1.252 :Note that all the CNAME records are only visible to machines within the LAN. If you want them visible to the outside world, you'll need to update the domain information with whoever hosts your DNS records for the wider world. :The reverse zone files, <code>/etc/bind/db.1.168.192</code> <code>/etc/bind/db.dddd.cccc.bbbb.aaaa</code> are like this (note the full stops at the end of the host definition records): ; ; BIND reverse data file for LAN, IPv4 ; $ORIGIN 1.168.192.IN-ADDR.ARPA. $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2007042701 ; Serial 1w ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 251 IN PTR desktop.domain.tld. 252 IN PTR server.domain.tld. @ IN NS localhost. ; ; BIND reverse data file for LAN, IPv6 ; $TTL 3D @ IN SOA server.domain.tld. root.domain.tld. ( 2022072401 ; Serial 1d ; Refresh 1d ; Retry 4w ; Expire 1w ) ; Negative Cache TTL ; ; define the authoritative name server NS server.domain.tld. NS desktop.domain.tld. ; our other hosts 1 IN PTR router.domain.tld. 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR router.domain.tld. 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR desktop.domain.tld. 2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR server.domain.tld. 3.4.4.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR www.domain.tld. 3.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR ns1.domain.tld. 4.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR ns2.domain.tld. 5.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR mail.domain.tld. 3.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0 PTR imap.domain.tld. 8.1.4.9.0.0.0.0.0.0.0.0.0.0.0.0 PTR git.domain.tld. :(Note the full stops after all the domain names.) * Update <code>/etc/network/interfaces</code> to tell <code>dnsmasq</code> about the nameservers # The primary network interface auto enp0s7 iface enp0s7 inet static address 192.168.1.251 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * Restart bind: root@server:~# systemctl restart bind9.service :and check that it works: root@server:~# host desktop.domain.tld root@server:~# host server.domain.tld server.domain.tld root@server:~# host www.google.com desktop.domain.tld :(and repeat for other hosts and both nameservers). You may need to reboot the master DNS server before the zone files transfer properly. == Chrooting the nameservers == I used to do this for security. But now AppArmor and SELinux are here, so I don't bother. It's just one more thing to go wrong. == Update the router == Finally, update the router to point to these nameservers, not the ones at the ISP. Normally, the router picks up the ISP's nameservers when it connects to the ISP. The router then acts as a recursive nameserver for the machines on the LAN, and it tells them when machines register with DHCP. You should be able to tell the router to use Server and Desktop as the nameservers it uses instead. Server and Desktop both know about the ISP's nameservers for finding IPs of machines outside the LAN. This means that the sequence of DNS requests is: # DHCP-connected machine asks the router for an IP number, # Router asks Server for the IP number, # Server asks the ISP's nameserver for the IP number, # ISP's nameserver finds it from somewhere and returns it. For DD-WRT, the settings to use are: * '''Router IP''' ** ''Local DNS:'' 192.168.0.252 * '''Network Address Server Settings (DHCP)''' ** ''Static DNS 1:'' 192.168.0.252 ** ''Static DNS 2:'' 192.168.0.251 ** ''Static DNS 3:'' 0.0.0.0 ** ''WINS:'' 0.0.0.0 ** ''Use DNSMasq for DHCP:'' OFF ** ''Use DNSMasq for DNS:'' OFF ** ''DHCP-Authoritative:'' OFF == See also == For a tutorial on how to set up nameservers, read [http://www.madboa.com/geek/soho-bind/ BIND for the Small LAN] by Paul Heinlein. Other sources are the book [http://www.netwidget.net/books/apress/dns/ Pro DNS and Bind], which is a great resource on the intricacies of BIND (especially [http://www.zytrax.com/books/dns/ch5/#fc2 Chapter 5 on BIND] and [http://www.zytrax.com/books/dns/ch6/ Chapter 6 giving example setups]). There is also the [https://help.ubuntu.com/community/BIND9ServerHowto Ubuntu BIND9 howto] and the Debian howtos on [http://www.howtoforge.com/debian_bind9_master_slave_system setting up a BIND server] and [http://www.howtoforge.com/howto_bind_chroot_debian chrooting a BIND server]. Finally, the <code>/usr/share/doc/bind9/README.Debian.gz</code> file on your machine's local disk has some good pointers. [[Category:Server setup]] cbc5e82b14ff16b29e7fa69e791520073f7f1080 0 1515 2975 2901 2021-04-13T14:04:49Z Neil 2 Removed NTP step wikitext text/x-wiki Here you'll find some notes I've written on how I set up my Linux machines. These notes are mainly for my own benefit. They're mainly just a series of steps to go through to get the desired results. They don't describe any design rationale for why I did things the way I did, and they don't describe a lot of the background information to make sense of the instructions. I have two Linux machines (one server, one desktop and a secondary server) and a few Windows and other machines. They're all connected into a single LAN, with both wired and wireless links, connected to a combined router and ASDL modem. These notes focus on setting up the two Linux machines, both running [http://www.ubuntu.com/ Ubuntu Linux]. These notes were written for Ubuntu 12.04 LTS and 13.10 (adapted from 8.04 LTS by way of 10.04 LTS), but should apply to other versions and distributions fairly easily. The two machines have the following setups: {| ! Server !! Desktop |- valign=top | * [[Router setup]] * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS master server]] * [[SSL certificate generation]] * [[Dovecot server setup|Mail server: IMAP (Dovecot)]] * [[Postfix server setup|Mail server: SMTP (Postfix)]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Web server setup|Web server ]] * [[Mediawiki farm setup|MediaWiki farm]] * [[Webmail setup|Secure webmail interface (Roundcube)]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup#Linux client setup|Scanner client]] * [[Git setup|Version control system server (Git)]] * [[Log viewer setup|Log statistics viewer (AWStats)]] * [[Backup setup|Automated backup system]] * [[Discourse setup|Discourse forum setup]] * [[Ghost setup|Ghost blog setup]] * [[Miscellaneous setup]] | * [[OS installation]] * [[Hostname and IP setup|Fixed hostname and IP number]] * [[Firewall setup|Firewall]] * [[Protect SSH]] * [[DNS server setup|DNS slave server]] * [[MySQL configuration]] * [[PostgreSQL configuration]] * [[Desktop mail forwarding]] * [[Web server setup|Web server]] * [[CUPS server setup|Printer server (CUPS)]] * [[Scanner setup|Scanner server]] * [[Samba setup|Samba server]] * [[Backup setup|Automated backup system]] * [[Miscellaneous setup]] * [[VNC setup]] * [[Cryptography setup]] * [[UPS setup]] |} Note that almost all of the instructions in these notes are given as the root user. Rather than preface every command with <tt>sudo</tt>, simply change user to root first: user@server:~# sudo -i and give your password. For more information, see the [https://help.ubuntu.com/8.04/serverguide/C/index.html Ubuntu Server Guide]. (For information on Ruby on Rails, see the [[Ruby on Rails setup]] page.) [[Category:Server setup]] e97c2c1de24751a34b0275c3a65dd77abab49823 0 1541 2976 2942 2021-04-13T14:06:06Z Neil 2 /* Update netplan */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Update netplan == Modify the file <code>/etc/netplan/01-network-manager-all.yaml</code> to include the stanza: network: version: 2 renderer: NetworkManager ethernets: eno1: dhcp4: no dhcp6: no addresses: [192.168.1.251/24] # <-- edit this line gateway4: 192.168.1.1 # <-- edit this line nameservers: search: [domain.tld] # <-- edit this line addresses: [192.168.1.251,192.168.1.252] # <-- edit this line Apply the changes root@desktop:~# netplan apply == Disable NetworkManager == Change <code>/etc/NetworkManager/NetworkManager.conf</code> so that <code>managed=true</code> and there's no <code>dns=dnsmasq</code> line active: [main] plugins=ifupdown,keyfile #dns=dnsmasq [ifupdown] managed=true == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Static IP numbers == * To give a network card a static IP number, add a stanza to <tt>/etc/network/interfaces</tt> auto eth0 iface eth0 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 dns-nameservers 192.168.1.252 192.168.1.251 dns-search domain.tld * To give a static IP to a WiFi connection using WPA Supplicant, use this interface stanza: iface wlan0 inet static address 192.168.1.101 netmask 255.255.255.0 network 192.168.1.0 broadcast 192.168.1.255 gateway 192.168.1.1 wpa-essid MyWifiID wpa-psk my-secret-key # wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf # iface default inet dhcp * Ensure that <code>/etc/resolv.conf</code> is a symbolic link to <code>/run/resolvconf/resolv.conf</code>. If it's still pointing to somewhere with <code>NetworkManager</code> in the name, delete it and recreate it: root@desktop:~# rm /etc/resolv.conf root@desktop:~# ln -s /run/resolvconf/resolv.conf /etc/resolv.conf (make sure the server and desktop get different IP numbers). Reboot the machines and they should both have their fixed IP numbers. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] 990058c512d0266180cf52be7589cc344597f6dc 2977 2976 2021-04-13T14:07:36Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == Update netplan == Modify the file <code>/etc/netplan/01-network-manager-all.yaml</code> to include the stanza: network: version: 2 renderer: NetworkManager ethernets: eno1: dhcp4: no dhcp6: no addresses: [192.168.1.251/24] # <-- edit this line gateway4: 192.168.1.1 # <-- edit this line nameservers: search: [domain.tld] # <-- edit this line addresses: [192.168.1.251,192.168.1.252] # <-- edit this line Apply the changes root@desktop:~# netplan apply == Disable NetworkManager == Change <code>/etc/NetworkManager/NetworkManager.conf</code> so that <code>managed=true</code> and there's no <code>dns=dnsmasq</code> line active: [main] plugins=ifupdown,keyfile #dns=dnsmasq [ifupdown] managed=true == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] a1b138740228e66664addabe4cb7782bbba51ce0 2996 2977 2022-07-23T14:20:18Z Neil 2 Updated for IPv6 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == IPv6 addressing plan == I couldn't find good guides to how to best use the huge space of IPv6 addresses. In the end I decided to use application-specific IP numbers, and when several applications are on the same machine, that machine has multiple addresses. My machines host multiple services for several domains so each combination gets its own IP number. == Update interfaces file == If the network is configured with <code>/etc/network/interfaces</code>, define the static IP numbers like this auto eno1 iface eno1 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 10.191.106.255 gateway 192.168.1.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 191.168.1.252 192.168.1.251 dns-search domain.tld iface env6 inet6 static netmask 64 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers aaaa:bbbb:cccc:dddd::8000:53 aaaa:bbbb:cccc:dddd::1000:54 dns-search domain.tld # njae ogedei iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:2 gateway aaaa:bbbb:cccc:dddd::1 # njae dns iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:53 # njae web iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:443 This uses the ''inherits'' feature to avoid repeating the DNS entries for each IPv6 address. Note that there should only be one <code>gateway</code> entry for both the IPv4 and IPv6 sides. Apply the changes root@desktop:~# systemctl restart networking.service == Update netplan == Modify the file <code>/etc/netplan/01-network-manager-all.yaml</code> to include the stanza: network: version: 2 renderer: NetworkManager ethernets: eno1: dhcp4: no dhcp6: no addresses: [192.168.1.251/24] # <-- edit this line gateway4: 192.168.1.1 # <-- edit this line nameservers: search: [domain.tld] # <-- edit this line addresses: [192.168.1.251,192.168.1.252] # <-- edit this line version: 2 renderer: NetworkManager ethernets: eno1: dhcp4: no dhcp6: no addresses: [192.168.1.251/24,aaaa:bbbb:cccc:dddd::1000:1/64,aaaa:bbbb:cccc:dddd::1000:53/64] routes: - to: default via: 192.168.1.1 - to: default via: aaaa:bbbb:cccc:dddd::1 nameservers: search: [domain.tld] addresses: [192.168.1.252,192.168.1.251,aaaa:bbbb:cccc:dddd::1000:53,aaaa:bbbb:cccc:dddd::1000:54] Apply the changes root@desktop:~# netplan apply == Disable NetworkManager == Change <code>/etc/NetworkManager/NetworkManager.conf</code> so that <code>managed=true</code> and there's no <code>dns=dnsmasq</code> line active: [main] plugins=ifupdown,keyfile #dns=dnsmasq [ifupdown] managed=true == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] 926c184540f120f393d03a0f8a0026eb54d9375a 2997 2996 2022-07-23T14:26:57Z Neil 2 /* Update netplan */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == IPv6 addressing plan == I couldn't find good guides to how to best use the huge space of IPv6 addresses. In the end I decided to use application-specific IP numbers, and when several applications are on the same machine, that machine has multiple addresses. My machines host multiple services for several domains so each combination gets its own IP number. == Update interfaces file == If the network is configured with <code>/etc/network/interfaces</code>, define the static IP numbers like this auto eno1 iface eno1 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 10.191.106.255 gateway 192.168.1.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 191.168.1.252 192.168.1.251 dns-search domain.tld iface env6 inet6 static netmask 64 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers aaaa:bbbb:cccc:dddd::8000:53 aaaa:bbbb:cccc:dddd::1000:54 dns-search domain.tld # njae ogedei iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:2 gateway aaaa:bbbb:cccc:dddd::1 # njae dns iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:53 # njae web iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:443 This uses the ''inherits'' feature to avoid repeating the DNS entries for each IPv6 address. Note that there should only be one <code>gateway</code> entry for both the IPv4 and IPv6 sides. Apply the changes root@desktop:~# systemctl restart networking.service == Update netplan == Modify the file <code>/etc/netplan/01-network-manager-all.yaml</code> to include the stanza: network: version: 2 renderer: NetworkManager ethernets: eno1: dhcp4: no dhcp6: no addresses: [192.168.1.251/24,aaaa:bbbb:cccc:dddd::1000:1/64,aaaa:bbbb:cccc:dddd::1000:53/64] routes: - to: default via: 192.168.1.1 - to: default via: aaaa:bbbb:cccc:dddd::1 nameservers: search: [domain.tld] addresses: [192.168.1.252,192.168.1.251,aaaa:bbbb:cccc:dddd::1000:53,aaaa:bbbb:cccc:dddd::1000:54] Apply the changes root@desktop:~# netplan apply == Disable NetworkManager == Change <code>/etc/NetworkManager/NetworkManager.conf</code> so that <code>managed=true</code> and there's no <code>dns=dnsmasq</code> line active: [main] plugins=ifupdown,keyfile #dns=dnsmasq [ifupdown] managed=true == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] a5028b043f02ba80561272f6e8c8d0f533ddcba0 3002 2997 2022-08-16T08:47:59Z Neil 2 /* Update interfaces file */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[OS installation]] |align="center" width="50%"| [[Firewall setup|Firewall]] |} The default setup for allocating IP numbers is dynamically, via DCHP from the router. That's fine for most machines, but server machines need fixed IP numbers. As both Linux machines act as servers, they both need fixed IP numbers. The first thing to do is to adjust the settings on the router so that some of the IP numbers on the LAN are not in the DHCP pool. While you're there, adjust the settings on the router to open various ports and pass the packets to the server. == Hostname == To give the Linux boxes their hostname, adjust the following files: * Change <code>/etc/hostname</code> to server * Change <code>/etc/hosts</code> to include 127.0.0.1 localhost 127.0.1.1 server.domain.tld server Find the hostname with <code>hostname</code>, the domain name with <code>dnsdomainname</code> and the FQDN with <code>hostname -f</code>. == IPv6 addressing plan == I couldn't find good guides to how to best use the huge space of IPv6 addresses. In the end I decided to use application-specific IP numbers, and when several applications are on the same machine, that machine has multiple addresses. My machines host multiple services for several domains so each combination gets its own IP number. == Update interfaces file == If the network is configured with <code>/etc/network/interfaces</code>, define the static IP numbers like this auto eno1 iface eno1 inet static address 192.168.1.252 netmask 255.255.255.0 network 192.168.1.0 broadcast 10.191.106.255 gateway 192.168.1.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 191.168.1.252 192.168.1.251 dns-search domain.tld iface env6 inet6 static netmask 64 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers aaaa:bbbb:cccc:dddd::8000:53 aaaa:bbbb:cccc:dddd::1000:54 dns-search domain.tld # domain server iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:2 gateway aaaa:bbbb:cccc:dddd::1 # domain dns iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:53 # domain web iface eno1 inet6 static inherits env6 address aaaa:bbbb:cccc:dddd::1000:443 This uses the ''inherits'' feature to avoid repeating the DNS entries for each IPv6 address. Note that there should only be one <code>gateway</code> entry for both the IPv4 and IPv6 sides. Apply the changes root@desktop:~# systemctl restart networking.service If you get an error, shown in <code>systemctl status networking.service</code> as <code>RTNETLINK answers: File exists</code>, it could be because you have more than one <code>gateway</code> statement for the IPv6 addresses. In that case, flush the interface and restart the network (you have to do this as on conjoined command): root@desktop:~# ip addr flush eno1 && systemctl restart networking.service == Update netplan == Modify the file <code>/etc/netplan/01-network-manager-all.yaml</code> to include the stanza: network: version: 2 renderer: NetworkManager ethernets: eno1: dhcp4: no dhcp6: no addresses: [192.168.1.251/24,aaaa:bbbb:cccc:dddd::1000:1/64,aaaa:bbbb:cccc:dddd::1000:53/64] routes: - to: default via: 192.168.1.1 - to: default via: aaaa:bbbb:cccc:dddd::1 nameservers: search: [domain.tld] addresses: [192.168.1.252,192.168.1.251,aaaa:bbbb:cccc:dddd::1000:53,aaaa:bbbb:cccc:dddd::1000:54] Apply the changes root@desktop:~# netplan apply == Disable NetworkManager == Change <code>/etc/NetworkManager/NetworkManager.conf</code> so that <code>managed=true</code> and there's no <code>dns=dnsmasq</code> line active: [main] plugins=ifupdown,keyfile #dns=dnsmasq [ifupdown] managed=true == Multiple network cards == One of my machines has two network cards. <tt>udev</tt> assigns interface names to the cards. The trouble is, it seems to do it randomly. To fix how the interface names are assigned to the NICs, modify the file <tt>/etc/udev/rules.d/70-persistent-net.rules</tt> to include the naming you want: # PCI device 0x11ab:0x4320 (skge) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0" # PCI device 0x10b7:0x9200 (3c59x) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="xx:xx:xx:xx:xx:xx", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1" :Remember to remove any existing naming rules. Note that you should replace the <tt>xx:xx:xx:xx:xx:xx</tt> with your cards' MAC addresses. == Changing hostnames == If you change the hostname after setting up the machine, you'll need to change a couple of things that encode the hostname. First is the OpenSSH server. The server keys will still be labelled with the old hostname. To regenerate the keys, delete them then reconfigure the <tt>openssh-server</tt> package: root@server:~# rm /etc/ssh/ssh_host_* root@server:~# rm dpkg-reconfigure -plow openssh-server You will then need to remove the key fingerprints from all machines that connect to this server, otherwise the SSH client will complain that the host it's connecting to has changed identity. For each user, remove the line for the given host in <tt>~/.ssh/known-hosts</tt>. When you next connect to the host, you'll be asked to confirm the key fingerprint again. (Taken from [http://www.cyberciti.biz/faq/howto-regenerate-openssh-host-keys/ Vivek Gite's instrutions].) The other thing is the user table in MySQL. MySQL creates users for the particular hostname. They're easily changed using the MySQL tool: root@desktop:~# mysql -u 'root' -p mysql> update mysql.user set host='newhost.domain.tld' where host='oldhost.domain.tld'; mysql> quit; ==External DNS records== I use [https://www.domaindiscount24.net DomainDiscount24] as a registrar for my domains. Their nameservers provide the canonical source for external DNS lookups for the <code>njae.me.uk</code> domain, among others. The domain settings there are fairly straightforward. I use their standard nameservers (<code>ns1.domaindiscount24.net</code>, <code>ns2.domaindiscount24.net</code>, and <code>ns3.domaindiscount24.net</code>). The DNS settings are: {| |- | IN A || 212.69.55.62 |- |rowspan="2" valign="top"| MX | 10 mail.njae.me.uk |- | 20 mail.microwavesushi.com |} Ensure the 'Make wildcard entry' box at the bottom of the page is checked, so that domains such as www.njae.me.uk and mail.njae.me.uk can be resolved. [[Category:Server setup]] 09a93a0bc4754721ed7dcb7af979e3277b200fe8 0 1526 2978 2949 2021-04-13T15:53:55Z Neil 2 /* Enable Calibre server */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) (Generate the package list with user@desktop:~$ comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) > manually-installed-packages.txt ) Note that many of them may already be installed from previous steps. === Snap packages === The command user@desktop:~$ snap list will list all the installed snap packages. So far, I have <code>discord</code>, <code>postman</code> and <code>slack</code> installed. === Downloads === Install these from normal downloads: * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://github.com/junegunn/fzf fzf] for fuzzy search * [https://www.sublimetext.com/docs/3/linux_repositories.html Sublime Text and Sublime Merge] * [https://snapcraft.io/slack Slack] as a Snap * [https://tresorit.com/download/linux Tresorit] for secure cloud backups VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) [[Category:Server setup]] c20f1719179f145abc9b552e5bdfc0745879de5f 2983 2978 2021-12-23T12:32:59Z Neil 2 /* Media server */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) (Generate the package list with user@desktop:~$ comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) > manually-installed-packages.txt ) Note that many of them may already be installed from previous steps. === Snap packages === The command user@desktop:~$ snap list will list all the installed snap packages. So far, I have <code>discord</code>, <code>postman</code> and <code>slack</code> installed. === Downloads === Install these from normal downloads: * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://github.com/junegunn/fzf fzf] for fuzzy search * [https://www.sublimetext.com/docs/3/linux_repositories.html Sublime Text and Sublime Merge] * [https://snapcraft.io/slack Slack] as a Snap * [https://tresorit.com/download/linux Tresorit] for secure cloud backups VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) * Install MiniDLNA: root@desktop:~# aptitude install minidlna * Edit the config file at <code>/etc/minidlna.conf</code> to have the settings: #network_interface=eth0 # Self-discovers if commented (good with NetworkManager) media_dir=A,/home/user/Music # Use A, P, and V to restrict media 'type' in directory media_dir=PV,/home/user/Pictures friendly_name=Desktp # Optional presentation_url=http://temujin.njae.me.uk/ db_dir=/var/cache/minidlna # Needs to be un-commented log_dir=/var/log # Needs to be un-commented inotify=yes # 'no' for less resources, restart required for new media * Open the firewall to allow DLNA root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8200 root@desktop:~# ufw allow in proto udp from 192.168.1.0/24 to any port 1900 * Restart everything root@desktop:~# systemctl force-reload minidlna.service root@desktop:~# systemctl restart minidlna.service root@desktop:~# ufw reload * Check it's working. Visit <code><nowiki>http://localhost:8200</nowiki></code> on the desktop and <code><nowiki>http://desktop.domain.tld:8200</nowiki></code> from another machine. [[Category:Server setup]] c187507cd0026c426535b1abee65759a0098ce7d 2984 2983 2021-12-26T12:33:02Z Neil 2 /* Media server */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Ghost setup|Ghost]] |align="center" width="50%"| [[VNC setup|VNC]] |} A couple of minor little things, more as personal taste than as functional changes. == Noisy boot == This is for the server machine. If I'm booting the server when it's attached to a monitor, it means there's something wrong with it. In that case, I want to see everything that's happening in the boot process. These instructions are for Grub-legacy. * Edit <code>/boot/grub/menu.lst</code> ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=quiet splash # defoptions= * Regenerate the Grub menu root@server:~# update-grub == Clean up Apt cache == Apt keeps copies of all the files it downloads. This can take up lots of disk space. <code>aptitude autoclean</code> discards the package files for obsolete package versions. Doing this weekly via a cron job stops the cache getting too big. Create <code>/etc/cron.weekly/aptitude-autoclean</code>: #! /bin/sh # aptitude-autoclean Cron script to clear apt cache of unneeded files. aptitude autoclean Then make the file executable: root@server:~# chmod a+x /etc/cron.weekly/aptitude-autoclean ==Remove old kernels== Old kernels hang around and clog up the <code>/boot</code> partition. Once you're sure the current kernel works fine, you can remove old kernels with this command: root@server:~# dpkg -l linux-* | awk '/^ii/{ print $2}' | grep -v -e `uname -r | cut -f1,2 -d"-"` | grep -v -e 'libc' | grep -e [0-9] | xargs aptitude remove == Install additional packages == This is mainly for the desktop machine: a few additional packages to install. === Normal packages === This is starting from a basic Kubuntu installation. First, install some general catch-all metapackages: root@desktop:~# aptitude install kde-full ubuntu-desktop Then install root@desktop:~# aptitude install kubuntu-restricted-extras ubuntu-restricted-extras :for MP3 and other playback: it should install a number of other things for you. Agree to removing <code>libavcodec53</code> and <code>libavutil51</code> and ensure that <code>libavcodec-extra-53</code> and <code>libavutil-extra-51</code> are installed. (The <code>-extra</code> packages supercede the original ones.) For DVD playback, install dvdcss: root@desktop:~# aptitude install libdvdread4 root@desktop:~# /usr/share/doc/libdvdread4/install-css.sh Then install all the packages in the [[additional packages list]]. Download the file as <code>packages-to-install.list</code> then install the packages with the command: root@desktop:~# aptitude install $(< packages-to-install.list ) (Generate the package list with user@desktop:~$ comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) > manually-installed-packages.txt ) Note that many of them may already be installed from previous steps. === Snap packages === The command user@desktop:~$ snap list will list all the installed snap packages. So far, I have <code>discord</code>, <code>postman</code> and <code>slack</code> installed. === Downloads === Install these from normal downloads: * [http://vue.tufts.edu/download/index.cfm?login=1 VUE] (registration required) (and [https://vue-forums.uit.tufts.edu/posts/list/848.page Linux .desktop and icon details]) * [http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ MongoDB] * [https://www.google.com/intl/en/chrome/browser/ Google Chrome] * [https://github.com/junegunn/fzf fzf] for fuzzy search * [https://www.sublimetext.com/docs/3/linux_repositories.html Sublime Text and Sublime Merge] * [https://snapcraft.io/slack Slack] as a Snap * [https://tresorit.com/download/linux Tresorit] for secure cloud backups VUE and Google Earth must be installed as a normal user. If you want them available to all users, you'll need to jiggle things around. Before installation, create a directory (somewhere like <code>/opt</code>) and make it world-writable. Then install the software into that directory as a normal user. VUE reports an error after installation, but it can be safely ignored. Acrobat Reader isn't in the official 13.10 repo, so you have to install it by hand. In addition, it only comes as a 32-bit binary, it it needs additional libraries. * Download the .deb file: [http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.5.5/enu/AdbeRdr9.5.5-1_i386linux_enu.deb] * Install some additional packages root@desktop:~$ aptitude install libxml2:i386 lib32stdc++6 libgtk2.0-0:i386 * Install the deb: root@desktop:~$ dpkg -i --force-architecture AdbeRdr9.5.5-1_i386linux_enu.deb && apt-get -f install === Firefox extensions === Add these Firefox extensions * Greasemonkey * Greasefire * NoScript * TabMixPlus * AdBlocke Plus * Better YouTube * Echofon (More details at the [http://kubuntuguide.org/Lucid Kubuntu Guide].) ==Install Ruby == * Install RMV as a multi-user environment: user@desktop:~$ curl -sSL https://get.rvm.io | sudo bash -s stable :Note that this command '''must''' be run as a non-root user with <code>sudo</code>. * Add users to the <code>rvm</code> group: user@desktop:~$ sudo adduser user rvm * Allow non-login shells to read the <code>rvm.sh</code> script. Add the following to each user's <code>~/.bashrc</code> file (including root's): if [ -f /etc/profile.d/rvm.sh ]; then source /etc/profile.d/rvm.sh fi * Add the same two lines to <code>/etc/bash.bashrc</code> so that newly created users get it added to their <code>~/.bashrc</code>s. :(Yes, this contradicts [http://rvm.io/integration/gnome-terminal the recommendation from RVM], but I think it's the more sensible place to put it.) * Reboot so that the new group permissions to take effect. * Install any additional packages: root@desktop:~# rvm requirements * Install the Rubies you want: user@desktop:~$ rvm list known user@desktop:~$ rvm install 2.1.0 user@desktop:~$ rvm install 1.9.3 * Make sure <code>bundler</code> is installed (note that it's done as <code>root</code>): root@desktop:~# gem install bundler * Add the following two lines to <code>/etc/rvmrc</code> rvm_project_rvmrc_default=1 rvm_autoinstall_bundler_flag=1 ===Starting a new Ruby project=== When creating a new project, add a <code>Gemfile</code> like this one: source 'https://rubygems.org' ruby "2.1.0" gem 'nokogiri' gem 'rack', '~>1.1' gem 'rspec', :require => 'spec' Then run user@desktop:~/path/to/project$ bundle install --binstubs See also the [[Ruby on Rails installation]]. See also: * [http://rvm.io/rvm/install RVM installation documentation] * [https://www.digitalocean.com/community/articles/how-to-install-ruby-on-rails-on-ubuntu-12-04-lts-precise-pangolin-with-rvm Installing Ruby with RVM on Ubuntu] * [http://leonard.io/blog/2013/10/installing-ruby-2.0.0-on-ubuntu-13-10-saucy-salamander/ Installing Ruby 2.0 without RVM] (for an alternative approach) ==Install Python3 and associated packages== * If you haven't already, install the basic packages: user@desktop:~$ sudo aptitude install python-all python3-all python-pip python3-pip * Then install various other scientific pacakges: user@desktop:~$ sudo aptitude install build-essential tk-dev libpng12-dev pandoc user@desktop:~$ sudo aptitude install python3-dev python3-tk python3-numpy python3-scipy python3-pandas user@desktop:~$ sudo aptitude install python-dev python-tk python-numpy python-scipy python-pandas python-sklearn user@desktop:~$ sudo aptitude install ipython ipython-notebook ipython3 ipython3-notebook user@desktop:~$ sudo aptitude install python-mpltoolkits.basemap python-pymongo python3-pymongo * Install Scikit-learn for Python3 user@desktop:~$ sudo pip3 install scikit-learn user@desktop:~$ sudo pip3 install jupyter jupyter_nbextensions_configurator jupyter_contrib_nbextensions user@desktop:~$ sudo jupyter nbextensions_configurator enable --system user@desktop:~$ sudo jupyter contrib nbextension install --system --symlink ===Install matplotlib basemap for Python3=== * Install the GEOS library user@desktop:~$ sudo aptitude install libgeos-dev * Download the [http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ Basemap tarball] * Unpack it and change to that directory user@desktop:~$ sudo python3 setup.py install * Check it user@desktop:~$ python3 Python 3.4.0 (default, Apr 11 2014, 13:05:11) [GCC 4.8.2] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from mpl_toolkits.basemap import Basemap >>> :(you're expecting no errors) ===Older instructions=== * Install pylinter: user@desktop:~$ sudo pip install pylint user@desktop:~$ sudo pip3 install pylint * Install IPython: user@desktop:~$ sudo pip install matplotlib ipython[all] user@desktop:~$ sudo pip3 install matplotlib ipython[all] If you want to use Virtual environments, you can use the built-in venv in Python3 or the add-on virtualenv for Python2. * Install Virtualenv and Virtualenvwrapper: user@desktop:~$ sudo pip install virtualenv virtualenvwrapper * Append these lines to <code>~/.bashrc </code> VIRTUALENVWRAPPER_PYTHON='/usr/bin/python' export WORKON_HOME=$HOME/.virtualenvs export PROJECT_HOME=$HOME/Documents/programming source /usr/local/bin/virtualenvwrapper.sh :Then source it user@desktop:~$ . ~/.bashrc * Create a virtual environment: user@desktop:~$ makevirtualenv pbasic :To use it, use the <code>workon</code> command, and use <code>deactivate</code> to stop using it: user@desktop:~$ workon pbasic (p3basic)user@desktop:~$ deactivate ===Alternative=== An alternative installation, using Pip not the Ubuntu package manager: http://bikulov.org/blog/2014/05/14/install-up-to-date-scientific-environment-in-ubuntu-14-dot-04-with-python-3-dot-4/ ==Install [https://github.com/SciRuby/iruby IRuby notebooks]== A bit more complex than it needs to be. * First, install all the compilation requirements, then finally IRuby root@desktop:~# aptitude install build-essential libtool-bin automake autoconf autogen root@desktop:~# aptitude install libczmq3 libczmq-dev root@desktop:~# gem update --system root@desktop:~# gem install bundler root@desktop:~# gem install rbczmq -- --with-system-libs root@desktop:~# gem install pry pry-doc awesome_print gnuplot rubyvis nyaplot root@desktop:~# gem install iruby * In the project you'll be working, create the <code>Gemfile</code> with contents: gem "iruby" gem 'rbczmq' gem 'ffi' gem 'pry' gem 'nyaplot' gem 'gnuplot' gem 'awesome_print' * Start IRuby with user@desktop:~$ iruby notebook == Install R language == I install the Bionic packages for R 3.5 from [https://cran.r-project.org/bin/linux/ubuntu/README.html Cran]. * Import the public key for these packages: root@desktop:~# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 51716619E084DAB9 * Install the package source list. Create the file <code>/etc/apt/sources.list.d/r-language-cran.list</code> to contain the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ * Install the R 3.5 files root@desktop:~# aptitude update root@desktop:~# aptitude install r-base ==Install extra system monitoring tools== * Install <code>htop</code>: root@server:~# aptitude install htop root@server:~# htop * Install <code>glances</code> (requires Python2): root@server:~# aptitude install python-dev root@server:~# pip install Glances PySensors root@server:~# glances == KMail server account == KMail doesn't seem to pick up the local machine name when sending mail. It seems to give the local host name as <code>desktop.localnet</code> rather than <code>desktop.domain.tld</code>. Therefore, when setting up the SMTP sending in KMail, set the local hostname in the 'Send custom hostname' field in the 'Advanced' tab of the SMTP transport settings dialog. == Emacs configuration == I like using Emacs, and here are a couple of simple tweaks to include in the <tt>~/.emacs</tt> configuration file: ;; .emacs ;;; uncomment this line to disable loading of "default.el" at startup ;; (setq inhibit-default-init t) ;; turn on font-lock mode (when (fboundp 'global-font-lock-mode) (global-font-lock-mode t)) ;; enable visual feedback on selections (setq transient-mark-mode t) ;; default to better frame titles (setq frame-title-format (concat "%b - emacs@" system-name)) ;; don't display the splash screen (setq inhibit-startup-message t) ;; Enable SLIME (setq inferior-lisp-program "/usr/bin/sbcl") (add-to-list 'load-path "/usr/share/emacs/site-lisp/slime") (require 'slime) (slime-setup) To run [http://common-lisp.net/project/slime/doc/html/ SLIME], type <code>M-x slime</code> in Emacs. == Password-less SSH logins == Let's say you want account <code>user@desktop.domain.tld</code> to be able to log into account <code>someone@server.domain.tld</code> without a password. Let's also assume that you can already SSH into <code>someone@server.domain.tld</code> with a password. * Log into <code>user@desktop.domain.tld</code> * Generate the keycode pair for this user: user@desktop:~$ ssh-keygen -t rsa :Accept the default location for the file (<code>~/.ssh/</code>) and just press &lt;Enter&gt; when asked for a passphrase. This has generated a private key (<code>~/.ssh/id_rsa</code>) and a public key (<code>~/.ssh/id_rsa.pub</code>). If you tell SSH on the <code>server</code> machine about the public key, the <code>server</code> machine will trust logins from that machine. This is done on a per-user basis. To allow a user '''here''' to log in as a '''someone''' over '''there''', do this as the user '''here''': user@here:~$ ssh-copy-id someone@there <code>user@here</code> should now be able to start SSH sessions as <code>someone@there</code> without giving <code>someone</code>'s password: user@desktop:~S ssh someone@there <login banner snipped> someone@there:~S Read [http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html detailed instructions] and [http://www.egenius.com/news_item.2008-06-18.6981552690 some clear instructions]. == Make Konqueror the default file manager == Ubuntu has chosen Dolphin as the default file manager for KDE. I prefer to use Konqueror, as I like the old-fashioned directory tree list. To make Konqueror default, use the <code>KDE Menu | System Settings | Default Applications | File Associations</code> and pick ''inode'' from the list in the window. For ''Directory'' and ''System Directory'', move <code>Konqueror</code> up the list to make it the preferred application for opening such things. == Install and use Sun Java == Use the instructions from the [http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html Web Upd8 team], as recommended on the [https://help.ubuntu.com/community/Java Ubuntu community documentation] site. root@server:~# add-apt-repository ppa:webupd8team/java root@server:~# aptitude update root@server:~# aptitude install oracle-java7-installer == Add KMenu actions == I like having the Quick Browser, Recent Documents, Settings, and Terminal Sessions actions on the KMenu. Enable them by editing <code>~/.kde/share/config/kickerrc</code> to include [menus] Extensions=recentdocs.desktop,prefmenu.desktop,konsolemenu.desktop,find.desktop NumVisibleEntries=6 RecentAppsStat=2 1227736562 ... and so on UseBrowser=true Other menu extensions can be found in <code>/usr/share/apps/kicker/menuext/</code> ==Install Virtualbox== Install the packages: root@desktop:~# aptitude install virtualbox virtualbox-guest-additions-iso * Note to self: Make sure the virtual machines get installed somewhere on /opt ==Enable the Compose key== The KDE Keyboard settings allow you to define the Compose key directly. For Ubuntu Unity (and perhaps Gnome), use <code>dconf</code> to set the compose key. First, find any bespoke keyboard settings: user@desktop:~$ dconf read /org/gnome/desktop/input-sources/xkb-options @as [] This says there are no settings currently. You can set the right Alt key (Alt GR) to be the compose key with: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['compose:ralt'] If there's are existing settings there, it looks like you need to include them in any dconf write, appending the new setting in the square brackets with settings separated by commas: user@desktop:~$ dconf write /org/gnome/desktop/input-sources/xkb-options ['other:option','compose:ralt'] Try these first and only muck around with <code>.Xmodmap</code> if that doesn't work for some reason. ===Change IBus default shortcut=== The default shortcut for IBus (to change the input method for different languages) is Ctrl-Space. This interfers with a number of applications, such as Emacs. To change the shortcut, either run <code>ibus-setup</code> as the relevant user, or click on the IBus icon in the system bar (a keyboard with a small globe on it). Change the default Ibus "next input method" shortcut to something obscure, such as Ctrl+Shift+Alt+Space. ===Disable IBus=== IBus's interception of keyboard signals interferes with some applications, notably Sublime Text. To disable IBus completely, run <code>im-config</code> as the user, click through the warnings about changing the input method, and select <code>none (do not set any IM from imconfig)</code> from the list, then press OK. You'll need to restart your KDE/X session for the changes to take effect. ===Enable XIM=== Enable the X Input Method for KDE and Gnome, rather than whatever they use as a default. This allows you to define additional Compose key codes. Add these lines to <code>~/.profile</code>: export GTK_IM_MODULE="xim" export QT_IM_MODULE="xim" then restart X. ===Overkill method using .Xmodmap=== First, find the keycodes for the keys you want to modify. Run <code>xev</code>, press the required keys, and note the key codes. I use Alt Gr as the Compose key, but don't have a spare key for Mode_switch. Edit <code>~/.Xmodmap</code> to have the lines keycode 113 = Mode_switch keycode 116 = Multi_key Insert the keycodes you have to suit. (If you don't have a <code>~/.Xmodmap</code> file, create it with user@desktop:~$ xmodmap -pke > ~/.Xmodmap ) Then run user@desktop:~$ xmodmap ~/.Xmodmap and the right Alt key (Alt Gr) should work and allow you access to funky additional characters. It allows you to combine multiple keystrokes for additional characters. Here is [http://en.wikipedia.org/wiki/Compose_key a list of Compose character codes]. ===Additional Compose sequences=== Now you can create the file <code>~/.XCompose</code> to include additional compose key sequences (a sample is below). Basic Greek letters in <code>.XCompose</code> from [http://shobute.com/posts/how-to-type-unicode-on-linux Shobute]; all the compose sequences you could think of at [https://github.com/kragen/xcompose Kragen's xcompose repo]. (Note that I've gone against tradition and use <code>g g</code> as the prefix for Greek, rather than <code>*</code>: that's because the asterisk is used for some over-rings on letters.) <pre><nowiki> include "%L" # EMOTICONS <Multi_key> <o> <underscore> <o> : "ಠ_ಠ" # LOOK OF DISAPPROVAL <Multi_key> <colon> <D> : "😃" # :D <Multi_key> <semicolon> <parenright> : "😉" # ;) <Multi_key> <colon> <p> : "😛" # :p <Multi_key> <semicolon> <p> : "😜" # ;p <Multi_key> <colon> <bar> : "😐" # :| <Multi_key> <colon> <apostrophe> <parenright> : "😢" # :'( # PUNCTUATION <Multi_key> <bracketright> <apostrophe> : "’" U2019 # RIGHT SINGLE QUOTATION MARK <Multi_key> <bracketleft> <apostrophe> : "‘" U2018 # LEFT SINGLE QUOTATION MARK <Multi_key> <bracketright> <quotedbl> : "”" U201d # RIGHT DOUBLE QUOTATION MARK <Multi_key> <bracketleft> <quotedbl> : "“" U201c # LEFT DOUBLE QUOTATION MARK <Multi_key> <space> <t> : " " U2009 # THIN SPACE <Multi_key> <space> <m> : " " U2003 # EM SPACE # Custom additions: Typography # <Multi_key> <period> <period> : "…" U2026 # HORIZONTAL ELLIPSIS <Multi_key> <backslash> <space> : "␣" U2423 # OPEN BOX # GREEK <Multi_key> <g> <g> <A> : "Α" U0391 # GREEK CAPITAL LETTER ALPHA <Multi_key> <g> <g> <a> : "α" U03B1 # GREEK SMALL LETTER ALPHA <Multi_key> <g> <g> <B> : "Β" U0392 # GREEK CAPITAL LETTER BETA <Multi_key> <g> <g> <b> : "β" U03B2 # GREEK SMALL LETTER BETA <Multi_key> <g> <g> <D> : "Δ" U0394 # GREEK CAPITAL LETTER DELTA <Multi_key> <g> <g> <d> : "δ" U03B4 # GREEK SMALL LETTER DELTA <Multi_key> <g> <g> <E> : "Ε" U0395 # GREEK CAPITAL LETTER EPSILON <Multi_key> <g> <g> <e> : "ε" U03B5 # GREEK SMALL LETTER EPSILON <Multi_key> <g> <g> <F> : "Φ" U03A6 # GREEK CAPITAL LETTER PHI <Multi_key> <g> <g> <f> : "φ" U03C6 # GREEK SMALL LETTER PHI <Multi_key> <g> <g> <G> : "Γ" U0393 # GREEK CAPITAL LETTER GAMMA <Multi_key> <g> <g> <g> : "γ" U03B3 # GREEK SMALL LETTER GAMMA <Multi_key> <g> <g> <H> : "Η" U0397 # GREEK CAPITAL LETTER ETA <Multi_key> <g> <g> <h> : "η" U03B7 # GREEK SMALL LETTER ETA <Multi_key> <g> <g> <I> : "Ι" U0399 # GREEK CAPITAL LETTER IOTA <Multi_key> <g> <g> <i> : "ι" U03B9 # GREEK SMALL LETTER IOTA <Multi_key> <g> <g> <J> : "Θ" U0398 # GREEK CAPITAL LETTER THETA <Multi_key> <g> <g> <j> : "θ" U03B8 # GREEK SMALL LETTER THETA <Multi_key> <g> <g> <K> : "Κ" U039A # GREEK CAPITAL LETTER KAPPA <Multi_key> <g> <g> <k> : "κ" U03BA # GREEK SMALL LETTER KAPPA <Multi_key> <g> <g> <L> : "Λ" U039B # GREEK CAPITAL LETTER LAMDA <Multi_key> <g> <g> <l> : "λ" U03BB # GREEK SMALL LETTER LAMDA <Multi_key> <g> <g> <M> : "Μ" U039C # GREEK CAPITAL LETTER MU <Multi_key> <g> <g> <m> : "μ" U03BC # GREEK SMALL LETTER MU <Multi_key> <g> <g> <N> : "Ν" U039D # GREEK CAPITAL LETTER NU <Multi_key> <g> <g> <n> : "ν" U03BD # GREEK SMALL LETTER NU <Multi_key> <g> <g> <O> : "Ο" U039F # GREEK CAPITAL LETTER OMICRON <Multi_key> <g> <g> <o> : "ο" U03BF # GREEK SMALL LETTER OMICRON <Multi_key> <g> <g> <P> : "Π" U03A0 # GREEK CAPITAL LETTER PI <Multi_key> <g> <g> <p> : "π" U03C0 # GREEK SMALL LETTER PI <Multi_key> <g> <g> <Q> : "Χ" U03A7 # GREEK CAPITAL LETTER CHI <Multi_key> <g> <g> <q> : "χ" U03C7 # GREEK SMALL LETTER CHI <Multi_key> <g> <g> <R> : "Ρ" U03A1 # GREEK CAPITAL LETTER RHO <Multi_key> <g> <g> <r> : "ρ" U03C1 # GREEK SMALL LETTER RHO <Multi_key> <g> <g> <S> : "Σ" U03A3 # GREEK CAPITAL LETTER SIGMA <Multi_key> <g> <g> <s> : "σ" U03C3 # GREEK SMALL LETTER SIGMA <Multi_key> <g> <g> <T> : "Τ" U03A4 # GREEK CAPITAL LETTER TAU <Multi_key> <g> <g> <t> : "τ" U03C4 # GREEK SMALL LETTER TAU <Multi_key> <g> <g> <U> : "Υ" U03A5 # GREEK CAPITAL LETTER UPSILON <Multi_key> <g> <g> <u> : "υ" U03C5 # GREEK SMALL LETTER UPSILON <Multi_key> <g> <g> <W> : "Ω" U03A9 # GREEK CAPITAL LETTER OMEGA <Multi_key> <g> <g> <w> : "ω" U03C9 # GREEK SMALL LETTER OMEGA <Multi_key> <g> <g> <X> : "Ξ" U039E # GREEK CAPITAL LETTER XI <Multi_key> <g> <g> <x> : "ξ" U03BE # GREEK SMALL LETTER XI <Multi_key> <g> <g> <Y> : "Ψ" U03A8 # GREEK CAPITAL LETTER PSI <Multi_key> <g> <g> <y> : "ψ" U03C8 # GREEK SMALL LETTER PSI <Multi_key> <g> <g> <Z> : "Ζ" U0396 # GREEK CAPITAL LETTER ZETA <Multi_key> <g> <g> <z> : "ζ" U03B6 # GREEK SMALL LETTER ZETA # Some archaic Greek. # "period" will indicate a sort of variant of some kind; g g is still the "greek" marker <Multi_key> <g> <g> <period> <p> : "ϖ" U03D6 # GREEK PI SYMBOL # Reserving .f in case we want PHI SYMBOL. Digamma was "w" sound anyway. <Multi_key> <g> <g> <period> <w> : "ϝ" U03DD # GREEK SMALL LETTER DIGAMMA <Multi_key> <g> <g> <period> <W> : "Ϝ" U03DC # GREEK CAPITAL LETTER DIGAMMA <Multi_key> <g> <g> <Q> : "Ϟ" U03DE # GREEK LETTER QOPPA <Multi_key> <g> <g> <q> : "ϟ" U03DF # GREEK SMALL LETTER QOPPA <Multi_key> <g> <g> <period> <Q> : "Ϙ" U03D8 # GREEK LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <period> <q> : "ϙ" U03D9 # GREEK SMALL LETTER ARCHAIC QOPPA <Multi_key> <g> <g> <ampersand> : "ϗ" U03D7 # GREEK KAI SYMBOL <Multi_key> <g> <g> <period> <Z> : "Ϡ" U03E0 # GREEK LETTER SAMPI <Multi_key> <g> <g> <period> <z> : "ϡ" U03E1 # GREEK SMALL LETTER SAMPI <Multi_key> <g> <g> <period> <period> <Z> : "Ͳ" U0372 # GREEK CAPITAL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <period> <z> : "ͳ" U0373 # GREEK SMALL LETTER ARCHAIC SAMPI <Multi_key> <g> <g> <period> <S> : "Ϛ" U03DA # GREEK LETTER STIGMA <Multi_key> <g> <g> <period> <s> : "ϛ" U03DB # GREEK SMALL LETTER STIGMA <Multi_key> <g> <g> <apostrophe> : "ʹ" U02B9 # MODIFIER LETTER PRIME, canonically equivalent to U0374 GREEK NUMERAL SIGN # While we're at it... <Multi_key> <g> <g> <period> <apostrophe> : "′" U2032 # PRIME <Multi_key> <g> <g> <period> <quotedbl> : "″" U2033 # DOUBLE PRIME <Multi_key> <g> <g> <comma> : "͵" U0375 # GREEK LOWER NUMERAL SIGN (for thousands) <Multi_key> <g> <g> <period> <b> : "ϐ" U03D0 # GREEK BETA SYMBOL <Multi_key> <g> <g> <period> <j> : "ϑ" U03D1 # GREEK THETA SYMBOL <Multi_key> <g> <g> <period> <Y> : "ϒ" U03D2 # GREEK UPSILON WITH HOOK SYMBOL <Multi_key> <g> <g> <period> <f> : "ϕ" U03D5 # GREEK PHI SYMBOL <Multi_key> <g> <g> <period> <k> : "ϰ" U03F0 # GREEK KAPPA SYMBOL <Multi_key> <g> <g> <period> <r> : "ϱ" U03F1 # GREEK RHO SYMBOL <Multi_key> <g> <g> <period> <U> : "ϴ" U03F4 # GREEK CAPITAL THETA SYMBOL <Multi_key> <g> <g> <period> <e> : "ϵ" U03F5 # GREEK LUNATE EPSILON SYMBOL <Multi_key> <g> <g> <period> <question> : "ϻ" U03FB # GREEK SMALL LETTER SAN <Multi_key> <g> <g> <period> <slash> : "Ϻ" U03FA # GREEK CAPITAL LETTER SAN </nowiki></pre> ==Install and configure Sublime Text== * Download [http://www.sublimetext.com/3 Sublime Text 3] and install it: root@desktop:~# dpkg -i sublime-text_build-3059_amd64.deb * When using KDE, update <code>~/.local/share/applications/mimeapps.list</code> to include the line: inode/directory=kde4-dolphin.desktop; :(This prevents Sublime Text from starting Nautilus when browsing files, and bringing most of Gnome with it.) * Install the [https://sublime.wbond.net/installation Package Control] package to install additional packages. * Use Package Control to install ** [https://sublime.wbond.net/packages/All%20Autocomplete All Autocomplete] ** [https://sublime.wbond.net/packages/Git Git] ** [https://sublime.wbond.net/packages/GitGutter Git Gutter] ** [https://sublime.wbond.net/packages/MarkdownEditing Markdown Editing] ** [https://sublime.wbond.net/packages/SideBarEnhancements Sidebar Enhancements] ** [https://sublime.wbond.net/packages/Theme%20-%20Soda Soda] (theme: UI colours) ** [https://sublime.wbond.net/packages/Tomorrow%20Color%20Schemes Tomorrow] (colour scheme: editor panel colours) :(Package suggestions from [http://dbader.org/blog/setting-up-sublime-text-for-python-development Daniel Bader].) * Enable the theme and colour scheme by opening the user preferences ''Preferences | Settings - User'' and adding these lines to <code>Packages/User/Preferences.sublime-settings</code>: { "color_scheme": "Packages/Tomorrow Color Schemes/Tomorrow-Night-Eighties.tmTheme", "font_face": "Source Code Pro", "font_size": 10, "ignored_packages": [ "Markdown", "Vintage" ], "soda_classic_tabs": true, "theme": "Soda Dark 3.sublime-theme" } :This also changes the default font. (Leave unchanged any other lines in that file.) * Open a Markdown file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Markdown.sublime-settings</code> { "color_scheme": "Packages/MarkdownEditing/MarkdownEditor-Dark.tmTheme" } * Install [https://sublime.wbond.net/packages/SublimeLinter Sublime Linter] and the linter packages for [https://sublime.wbond.net/packages/SublimeLinter-pylint Python], [https://sublime.wbond.net/packages/SublimeLinter-ruby Ruby], [https://sublime.wbond.net/packages/SublimeLinter-ghc Haskell], and [https://sublime.wbond.net/packages/SublimeLinter-json JSON]. You'll also need to install the <code>pylint</code> package via <code>pip</code>. :To make the Python linter use Python 3 by default, insert the <code>"@python": 3</code> line into ''Preferences | Package Settings | SublimeLinter | Settings - User'' (<code>Packages/User/SublimeLinter.sublime-settings</code>): { "user": { "linters": { "pylint": { "@python": 3 // Insert this line } } } } * Open a Python file, select ''Preferences | Settings - More | Syntax specific - User'', and add these lines to <code>Packages/User/Python.sublime-settings</code> { "rulers": [80] } ==Enable KSysGuard on the server== KSysGuard is a nice graphic system performance monitor. * On the server, install <code>ksysguardd</code>, a lightweight monitor server that doesn't require all of KDE. root@server:~# aptitude install ksysguardd * Create a user for ksysguard to run under: root@server:~# groupadd ksysguard root@server:~# useradd -s /usr/bin/ksysguardd -g ksysguard ksysguard root@server:~# mkdir -p /home/ksysguard/.ssh root@server:~# chown -R ksysguard:ksysguard /home/ksysguard :(the <code>useradd</code> line means that the <code>ksysguard</code> user will run the <code>ksysguardd</code> monitor as a login shell rather than Bash. * Copy across the user's ssh public key to allow passwordless logins user@desktop:~# scp ~/.ssh/id_rsa.pub user@desktop:ksysguard_key root@server:~# mv /home/user/ksysguard_key /home/ksysguard/.ssh/authorized_keys root@server:~# chown ksysguard:ksysguard /home/ksysguard/.ssh/authorized_keys * Add ksysguard to the list of users that can connect via ssh. Add <code>ksysguard</code> to the end of the <code>AllowUsers</code> line in <code>/etc/ssh/sshd_config</code>. * Test it works by logging into the ksysguard user user@desktop:~$ ssh ksysguard@ogedei ksysguardd 4 (c) 1999, 2000, 2001, 2002 Chris Schlaeger <cs@kde.org> (c) 2001 Tobias Koenig <tokoe@kde.org> (c) 2006-2008 Greg Martyn <greg.martyn@gmail.com> This program is part of the KDE Project and licensed under the GNU GPL version 2. See http://www.kde.org for details. ksysguardd> :the commands <code>monitors</code> and <code>quit</code> should be helpful. * In the <code>ksysguard</code> graphical client, select File | Monitor Remote Machine and give the details of the server. You should now be able to add monitors for the server. Instructions taken from [http://buffalo.nas-central.org/wiki/KDE_ksysguardd_on_the_Linkstation NRS Central]. ==Enable Calibre server== Calibre is an ebook management system. It also supports the OPDS protocol for sharing ebooks across a network. ===Get it working=== * Open the firewall to port 8080 on the local network root@desktop:~# ufw allow from 192.168.1.0/24 to any port 8080 * Start the server: user@desktop:~$ calibre-server ~/Documents/calibre-library/ * Test that you can see the library with a browser pointed to <code>localhost:8080</code> and that OPDS-aware apps (such as FBReader on Android) can see the library. * Kill that server instance ===Create systemd job=== Create the file <code>/lib/systemd/system/calibre-server.service</code> (from version 3.0 onwards) [Unit] Description=Calibre ebook content service [Service] ExecStart=/usr/bin/calibre-server --port 8080 /home/user/path/to/calibre-library [Install] WantedBy=multi-user.target (The more correct way to do this is to create the file <code>/etc/sysconfig/calibre-server</code> that contains the environment variables (such as path to the library and port to serve the library on, and include that environment with <code>EnvironmentFile=etc/sysconfig/calibre-server</code> . But it's hardly worth it in this case.) Reload the Systemd configuration and start the server: root@desktop:~# systemctl enable calibre-server.service root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl restart calibre-server.service root@desktop:~# systemctl status calibre-server * Check again that the Calibre server is running. ==Change tooltip colours== I use KDE as an environemnt, with a "dark text on light background" theme. The default Ubuntu theme has tooltips of "light text on dark background." Some non-KDE apps pick up the "light text" part of tooltips, but use the "light background" setting from KDE. This makes tooltips in several apps (GIMP and Inkscape, among others) unusable. Fix this by editing <code>~/.gtkrc-2.0</code> to include, at the end: style "gnome-color-chooser-tooltips" { bg[NORMAL] = "#FFFFAF" fg[NORMAL] = "#000000" } widget "gtk-tooltip*" style "gnome-color-chooser-tooltips" Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/gtk.css</code> so that the tooltip colours are: tooltip_bg_color #FFFFAF; tooltip_fg_color #000000; Edit <code>/usr/share/themes/Ambiance/gtkrc-3.0/settings.ini</code> so that the <code>gtk-color-scheme</code> colours are: tooltip_bg_color:#FFFFAF tooltip_fg_color:#000000 :(you may have to dig through that setting (Instructions from [http://www.iwillfolo.com/how-to-change-gtk-apps-tooltips-text-and-background-color/ I Will Folo].) ==Media server== Install MiniDLNA for uPNP/DLNA serving of media across the network. (Instructions to follow.) * Install MiniDLNA: root@desktop:~# aptitude install minidlna * Edit the config file at <code>/etc/minidlna.conf</code> to have the settings: #network_interface=eth0 # Self-discovers if commented (good with NetworkManager) media_dir=A,/home/user/Music # Use A, P, and V to restrict media 'type' in directory media_dir=PV,/home/user/Pictures friendly_name=Desktp # Optional presentation_url=http://temujin.njae.me.uk/ db_dir=/var/cache/minidlna # Needs to be un-commented log_dir=/var/log # Needs to be un-commented inotify=yes # 'no' for less resources, restart required for new media * Open the firewall to allow DLNA root@desktop:~# ufw allow in proto tcp from 192.168.1.0/24 to any port 8200 root@desktop:~# ufw allow in proto udp from 192.168.1.0/24 to any port 1900 * Restart everything root@desktop:~# systemctl force-reload minidlna.service root@desktop:~# systemctl restart minidlna.service root@desktop:~# ufw reload * Check it's working. Visit <code><nowiki>http://localhost:8200</nowiki></code> on the desktop and <code><nowiki>http://desktop.domain.tld:8200</nowiki></code> from another machine. * And finally, check that the <code>minidlna</code> user can read all the media files! [[Category:Server setup]] a8f5e364d001914f527be51d18e3c7e97bebb304 0 1516 2979 2431 2021-05-18T15:46:55Z Neil 2 Added NVMe boot section wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Router setup]] |align="center" width="50%"| [[Hostname and IP setup|Hostname and IP]] |} Installation of the operating systems is straightforward. The server has [http://www.ubuntu.com/business/server/overview Ubuntu Server edition] installed and the desktop machine has [http://www.kubuntu.org/ Kubuntu] (no reason for selecting KDE, other than it's what I'm used to). The installation scripts for the server should ask if you want to install a standard LAMP server; you do. The server needs the additional packages <code>build-essentials</code> (for basic software development) and <code>openssh-server</code> (as the server will be run as a headless box). Other necessary packages will be mentioned with the relevant setup. ==Booting from an NVMe drive== My desktop machine has an NVMe drive as the main storage with various SSDs for documents, backups, and the like. The main OS is on the NVMe drive. Unfortunately, the motherboard can't boot from that drive, and the system uses UEFI booting. That requires a bit of fiddling around to get working. I ended up with this pattern: * /dev/sdb4 , mounted on /boot/efi , Fat32 filesystem. The partition has the '''boot''' and '''esp''' flags. 1Gb should be plenty. * /dev/sdb1 , mounted on /boot , ext4 filesystem. The partition has the '''legacy_boot''' flag. A few Gb is all that's needed. * /dev/nvme0n1 , mounted on / , ext4 filesysem. This is the main partition of the OS and files. This pattern came from trial and error, following the guidance from [https://delightlylinux.wordpress.com/2017/12/21/system-boot-when-using-nvme/ System Boot When Using NVMe] and [https://askubuntu.com/questions/1074098/booting-a-nvme-drive Booting a NVMe drive]. ==Partitioning the root filesystem (with LVM)== For both setups, I don't bother doing anything complicated with disk partitioning. I have the recommended size partition for swap, and allocated a few GB each for <code>/var/log</code> and <code>/tmp</code>, and everything else just goes into one large partition. However, for reasons that aren't that clear any more, I couldn't get the installer to create those partitions, so I had to do it manually. This was complicated by the fact that I wanted to resize the root filesystem, which you can't do when it's mounted. The way around it is to boot the system, after installation, with the Ubuntu Live CD. (This requires that the machine has a keyboard, mouse, and monitor attached. Something to bear in mind if you're installing Ubuntu on a headless server.) Once you've started Ubuntu, use the "Try Ubuntu" option. * Open a terminal and install the LVM tools: user@subuntu:~$ sudo -i root@ubuntu:~# apt-get install lvm2 * Find the name of the root filesystem LVM: root@ubuntu:~# lvs root@ubuntu:~# vgchange --available y server * Resize the logical volume containing the root filesystem, including resizing the filesystem there: lvresize --resizefs --size -50G /dev/server/root * Now you have some space, you can resize the swap logical volume root@ubuntu:~# swapoff -v /dev/server/swap root@ubuntu:~# lvresize /dev/server/swap -L +2G root@ubuntu:~# mkswap /dev/server/swap root@ubuntu:~# swapon -va :(You won't need the <code>swapon</code> and <code>swapoff</code> commands if you're doing it from the LiveCD.) * Create two new logical volumes, for <code>/var/log</code> and <code>/tmp</code>, and add filesystems to them: root@ubuntu:~# lvcreate -L 20G -n tmp server root@ubuntu:~# lvcreate -L 25G -n varlog server root@ubuntu:~# mkfs.ext4 /dev/server/tmp root@ubuntu:~# mkfs.ext4 /dev/server/varlog * Create mount points and mount the <code>root</code>, <code>varlog</code> partition, and <code>tmp</code> partitions: root@ubuntu:~# mkdir /media/rootfs root@ubuntu:~# mount /dev/server/root root@ubuntu:~# mkdir /media/varlog root@ubuntu:~# mount /dev/server/varlog root@ubuntu:~# mkdir /media/tmp root@ubuntu:~# mount /dev/server/tmp * Copy the existing contents of <code>/var/log</code> to the new logical volume: root@ubuntu:~# cp -pr basefs/var/log/* varlog/ * Modify <code>/etc/fstab</code> to use the new volumes. Add these lines immediately after where the root filesystem is mounted: /dev/mapper/server-tmp /tmp ext4 defaults 0 1 /dev/mapper/server-varlog /var/log ext4 defaults 0 1 * Remove the LiveCD and reboot. * After booting, change the permissions on /tmp: chmod a+rwx /tmp chmod o+t /tmp :(You may have problems with [[MySQL configuration|MySQL]], so that my need reinstalling.) (See more on [http://linuxconfig.org/Linux_lvm_-_Logical_Volume_Manager#5-create-logical-volumes LVM management] and [http://www.centos.org/docs/4/4.5/System_Administration_Guide/Adding_Swap_Space-Extending_Swap_on_an_LVM2_Logical_Volume.html Extending swap on LVM].) == Install proprietary graphics card driver == I have a NVidia GeForce G210 video card. The proprietary video card driver uses the card's GPU for better graphics; the open source one makes KDE hang all too often. This means you need to install the proprietary driver as soon as the OS is installed on the desktop machine. To check that the card is supported, do: root@desktop:~# lspci | grep -i nvidia 02:00.0 VGA compatible controller: nVidia Corporation GT218 [GeForce G210] (rev a2) note the code at the start of the line, and use that to find the PCI ID of the card: root@desktop:~# lspci -n | grep 02:00 02:00.0 0300: 10de:0a60 (rev a2) and check that the PCI ID you get is in the [http://us.download.nvidia.com/XFree86/Linux-x86/169.12/README/appendix-a.html list of supported products] (Mine's <code>10de:0a60</code>; the GeForce 6600 on the old machine was <code>10de:00f2</code>). Installing the driver is easy. Go to KMenu -> System -> Hardware Drivers Manager (in Kubuntu 10.04, it's K -> Applications -> System -> Hardware Drivers) and check the box to enable the restricted drivers for the NVIDIA card. Adept will then download the right package for you. Restart X (Rebooting the machine is probably easier) and all should be fine. Full details are at the [https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia Nvidia driver how to] on the [https://help.ubuntu.com/ Ubuntu documentation site]. [[Category:Server setup]] d5c3e3b79272cd7ea1d73e5a7aa9fa61ee72eccd 0 1648 2981 2866 2021-10-01T14:30:20Z Neil 2 wikitext text/x-wiki I have an APC-made UPS (uninterruptable power supply). It comes with the ability for connected PCs to monitor its state. I have it connected by USB cable to the desktop machine. That machine communicates with the server, so they are both aware of power status. ==Installation== Simply follow the [https://help.ubuntu.com/community/apcupsd Ubuntu instructions]. The desktop machine has <code>/etc/apcupsd/apcupsd.conf</code> containing: UPSCABLE usb UPSTYPE usb DEVICE POLLTIME 60 NISIP 0.0.0.0 The server machine has <code>/etc/apcupsd/apcupsd.conf</code> UPSCABLE ether UPSTYPE net DEVICE desktop.domain.tld:3551 POLLTIME 10 The desktop machine will require port 3551 open on its [[Firewall setup|firewall]]. On both machines, set <code>ISCONFIGURED=yes</code> in <code>/etc/default/apcupsd</code> and restart the monitor. root@server:~# systemctl restart apcupsd.service root@server:~# apcaccess status Finally, configure the BIOS on all machines to start automatically when power is supplied. == See also == * [https://help.ubuntu.com/community/apcupsd Ubuntu instructions on apcupsd] * [http://www.apcupsd.org/manual/manual.html#nis-server-client-configuration-using-the-net-driver acpupsd documentation]. [[Category:Server setup]] 96d4f5056e3fdb819082a11b974b441e7b887430 0 1540 2986 2952 2022-05-23T09:55:47Z Neil 2 Added Borg section, just notes wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly Then restart the <code>cron</code> daemon. root@desktop:~# systemctl restart cron.service === SystemD version === The timing of Anacron is controlled by SystemD, as specified in the <code>/etc/systemd/system/timers.target.wants/anacron.timer</code> file. Change the file to have the updated <code>OnCalendar</code> line: [Unit] Description=Trigger anacron every hour [Timer] # OnCalendar=*-*-* 07..23:30 OnCalendar=*-*-* 01:03 RandomizedDelaySec=5m Persistent=true [Install] WantedBy=timers.target Then restart the <code>anacron</code> daemon. root@desktop:~# systemctl restart anacron.service == Borg == : Just notes at the moment. * [https://borgbackup.readthedocs.io/en/stable/quickstart.html# Borg], and [https://practical-admin.com/blog/backups-using-borg/ a guide] and [https://blog.andrewkeech.com/posts/170718_borg.html another guide]. * Create backup users on each machine. Give them login shells, home directories. Initially create passwords. * Create ssh keys for each * ssh-copy-id for backup and root onto each machine. * Check permissions of .ssh and .ssh/authorized_keys * Delete passwords * Systemd scripts in /lib/systemd/system/ * systemctl daemon-reload * systemctl enable borg_backup_to_desktop.timer * systemctl restart timers.target [[Category:Server setup]] 74a98f76b0b4bee0a220f97d863bc039bcc327b0 2987 2986 2022-05-24T13:03:49Z Neil 2 /* Borg */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I have a simple process for having my two machines back up themselves every night. This is kicked off with a <tt>cron</tt> script. Personally, I do full backups every time (rather than incremental backups) as it makes recovery easier. == Create users == Backups are safer if they use non-privileged users to do the backups. For instance, create the <tt>backup</tt> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Desktop backup == On the desktop machine, there is <code>/etc/cron.daily/makedailybackups</code>: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ # chmod 600 "$BACKUPFILENAME.music.tar.bz2") & # (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ # chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.pictures.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 2 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +1 -exec rm -f {} \; and <code>/etc/cron.monthly/makemonthlybackups</code> #!/bin/bash BACKUPFILENAME="/backup/monthly-backup.desktop.$(date -I)" # Create new backup files. Do them in subshells so they get done in parallel (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ chmod 600 "$BACKUPFILENAME.home.tar.bz2" ) & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.music.tar.bz2" /opt/music ; \ chmod 600 "$BACKUPFILENAME.music.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.sources.tar.bz2" /opt/sources ; \ chmod 600 "$BACKUPFILENAME.sources.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.photos.tar.bz2" /opt/photos ; \ chmod 600 "$BACKUPFILENAME.photos.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc ; \ chmod 600 "$BACKUPFILENAME.etc.tar.bz2") & (tar --create --absolute-names --bzip --file="$BACKUPFILENAME.chroot.tar.bz2" /chroot ; \ chmod 600 "$BACKUPFILENAME.chroot.tar.bz2" ) & wait # Remove backups more than 40 days old find /backup/ -maxdepth 1 -name 'monthly-backup*' -type f -mtime +40 -exec rm -f {} \; In either case, a file is restored with the command root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx Adding the option <code>--absolute-names</code> would restore the files to paths anchored at root. To extract specific files, use root@desktop:~# tar --extract --same-owner --preserve-permissions --keep-old-files --verbose --bzip --file=xxx --wildcards --no-anchored 'make*backup' which will extract files that match the given pattern (<code>--wildcards</code>), regardless of their preceeding path name (<code>--no-anchored</code>). == Server backup == Backups on the server are a little more complicated, because of all the various services the server provides. Here's <code>/etc/cron.daily/makedailybackups</code>, the daily backup script: #!/bin/bash BACKUPFILENAME="/backup/daily-backup.ogedei.$(date -I)" # Remove backups more than 4 days old find /backup/ -maxdepth 1 -name 'daily-backup*' -type f -daystart -mtime +5 -exec rm -f {} \; # Create file backups nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.opt.tar.bz2" \ --exclude='/opt/svn/*' /opt nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.etc.tar.bz2" /etc nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.home.tar.bz2" \ --exclude='/home/git/*' /home nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.vmail.tar.bz2" /var/vmail nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.git.tar.bz2" /home/git nice tar --create --absolute-names --bzip --file="$BACKUPFILENAME.www.tar.bz2" /var/www # Create database backups for db in njaewikidb mkrpgwikidb ; do nice mysqldump --user='backup' --opt --databases $db | nice bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" chmod 400 "${BACKUPFILENAME}.${db}-dump.sql.bz2" done pg_dump -U backup -Fc dmarc > "${BACKUPFILENAME}.dmarc-dump" chmod 400 "${BACKUPFILENAME}.dmarc-dump" for f in "$BACKUPFILENAME.opt.tar.bz2" "$BACKUPFILENAME.etc.tar.bz2" "$BACKUPFILENAME.home.tar.bz2" \ "$BACKUPFILENAME.vmail.tar.bz2" "$BACKUPFILENAME.git.tar.bz2" "$BACKUPFILENAME.www.tar.bz2" \ "$BACKUPFILENAME.njaewikidb-dump.sql.bz2" "$BACKUPFILENAME.mkrpgwikidb-dump.sql.bz2" \ "$BACKUPFILENAME.dmarc-dump" do chmod 400 $f nice scp $f user@desktop.domain.tld:/backup ssh user@desktop.domain.tld "/backup/lockdown $f" done To restore MySQL dumps, you may need to create the database in MySQL: root@server:~# mysql -u 'root'@'localhost' -p -e 'create database DB_NAME' Then get MySQL to run it: root@server:~# bunzip2 DB_DUMP_FILE | mysql --user='root' -p To restore Postgres dumps, drop the database then restore from the backup dump: root@server:~# sudo -u postgres dropdb DATABASE root@server:~# sudo -u postgres pg_restore -Fc -C -d postgres DB_DUMP_FILE == Cron (and Anacron) timing == I found that the default timing for kicking off cron jobs (at around 7.30) meant that the backups were still being created when I started work. That was easily fixed by altering <code>/etc/crontab</code> on the server so that the ''hours'' column (the second column) is 1 instead of 7: 25 1 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 1 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 1 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) On the desktop machines, alter <code>/etc/cron.d/anacron</code> to contain: 30 1 * * * root test -x /etc/init.d/anacron && /usr/sbin/invoke-rc.d anacron start >/dev/null If needed, edit the contents of the files in <code>/var/spool/anacron/</code> to make weekly and monthly cron jobs happen on sensible days. Use <code>touch</code> to adjust the datestamps to the same dates. root@desktop:~# touch -m -t 201005010100 /var/spool/anacron/cron.monthly Then restart the <code>cron</code> daemon. root@desktop:~# systemctl restart cron.service === SystemD version === The timing of Anacron is controlled by SystemD, as specified in the <code>/etc/systemd/system/timers.target.wants/anacron.timer</code> file. Change the file to have the updated <code>OnCalendar</code> line: [Unit] Description=Trigger anacron every hour [Timer] # OnCalendar=*-*-* 07..23:30 OnCalendar=*-*-* 01:03 RandomizedDelaySec=5m Persistent=true [Install] WantedBy=timers.target Then restart the <code>anacron</code> daemon. root@desktop:~# systemctl restart anacron.service == Borg == : Just notes at the moment. * [https://borgbackup.readthedocs.io/en/stable/quickstart.html# Borg], and [https://practical-admin.com/blog/backups-using-borg/ a guide] and [https://blog.andrewkeech.com/posts/170718_borg.html another guide]. * Create backup users on each machine. Give them login shells, home directories. Initially create passwords. * Create ssh keys for each * ssh-copy-id for backup and root onto each machine. * Check permissions of .ssh and .ssh/authorized_keys * Delete passwords * Systemd scripts in /lib/systemd/system/ * systemctl daemon-reload * systemctl enable borg_backup_to_desktop.timer * systemctl restart timers.target Extracting files, probably easier with <code>borg mount</code> [[Category:Server setup]] 53023464ba89674dbda6db858559ccb750117433 2988 2987 2022-06-11T11:21:13Z Neil 2 wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I use [https://borgbackup.readthedocs.io/en/stable/ Borg] for taking backups of my two machines every night. It's controlled by a SystemD timer. The deduplication of backups allows many backups to be made in not much space. Borg allows easy recovery by allowing archives to be mounted as a drive. The desktop and server machine each have two repositories. Each machine creates a backup locally and one on the other machine. I do it this way so there are no clashes with concurrent backups writing to the same repository from different machines. == Create users == Backups are safer if they use non-privileged users to do the backups. I create these users for the OS and databases. === OS=== On each machine, create <code>backup</code> users. These will be used as the target of backups to remote machines. Give these users login shells and home directories. For each user, create an SSH key and transfer it to the other machine with <code>ssh-copy-id</code>. Once you've set up that both <code>root</code> and <code>backup</code> on each machine can SSH into <code>backup</code> on the other machine, you can disable the passwords for each <code>backup</code> user. === MySQL === For instance, create the <code>backup</code> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. === Postgres === For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Creating archives == As the <code>backup</code> user, create two repositories on each machine, one for each machine being backed up. Remember to check and write down the repository encryption keys! root@desktop:~# mkdir -p /backup/from_desktop root@desktop:~# mkdir -p /backup/from_server root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_desktop root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_server Do the same on the server, creating the two repositories. On each machine, create the file <code>/etc/borg/backup_patterns</code> that defines what files are backed up. This allows the same files to be backed up to each destination. P sh R /home/ - /home/*/.cache - /home/*/TresoritDrive - /home/*/.kde/share/apps/nepomuk - /home/*/.thunderbird - /home/*/.local/share - /home/*/.rvm - /home/*/.vagrant.d - /home/*/anaconda3 - /home/*/.ghcup - /home/**/machine-learning-datasets R /opt/photos R /opt/music R /opt/sources R /etc R /lib/systemd R /var/www To do the backups from the desktop, create the files <code>/etc/borg/backup_to_desktop</code> and <code>/etc/borg/backup_to_server</code>. Below is the content of <code>/etc/borg/backup_to_desktop</code>. <code>/etc/borg/backup_to_server</code> is identical except for the line at the top which becomes <code>export BORG_REPO=backup@server:/backup/from_desktop</code> and the appropriate passphrase. #!/bin/sh # the repo and its passphrase export BORG_REPO=/backup/from_desktop export BORG_PASSPHRASE='long complex passphrase' # some helpers and error handling: info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; } trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM info "Starting backup" # backup the directories borg create \ --verbose --filter AME \ --list --stats --show-rc \ --one-file-system \ --compression zstd,22 --exclude-caches \ --patterns-from /etc/borg/backup_patterns \ ::'{hostname}-daily-{now}' 2>&1 backup_exit=$? info "Pruning repository" # prune the repo borg prune \ --list \ --prefix '{hostname}-daily-' \ --show-rc \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 24 2>&1 prune_exit=$? # Compact the repo, for when version 1.2 is available # borg compact # compact_exit=$? # use highest exit code as exit code global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit )) # global_exit=$(( compact_exit > global_exit ? compact_exit : global_exit )) if [ ${global_exit} -eq 1 ]; then info "Backup, Prune, or Compact finished with a warning" fi if [ ${global_exit} -gt 1 ]; then info "Backup, Prune, or Compact finished with an error" fi exit ${global_exit} == Services and timers == === SystemD version === The timing of Anacron is controlled by SystemD, as specified in the <code>/etc/systemd/system/timers.target.wants/anacron.timer</code> file. Change the file to have the updated <code>OnCalendar</code> line: [Unit] Description=Trigger anacron every hour [Timer] # OnCalendar=*-*-* 07..23:30 OnCalendar=*-*-* 01:03 RandomizedDelaySec=5m Persistent=true [Install] WantedBy=timers.target Then restart the <code>anacron</code> daemon. root@desktop:~# systemctl restart anacron.service == Borg == : Just notes at the moment. * [https://borgbackup.readthedocs.io/en/stable/quickstart.html# Borg], and [https://practical-admin.com/blog/backups-using-borg/ a guide] and [https://blog.andrewkeech.com/posts/170718_borg.html another guide]. * Create backup users on each machine. Give them login shells, home directories. Initially create passwords. * Create ssh keys for each * ssh-copy-id for backup and root onto each machine. * Check permissions of .ssh and .ssh/authorized_keys * Delete passwords * Systemd scripts in /lib/systemd/system/ * systemctl daemon-reload * systemctl enable borg_backup_to_desktop.timer * systemctl restart timers.target Extracting files, probably easier with <code>borg mount</code> [[Category:Server setup]] 745e896d9c281ba562a61eab45778b38af1d4853 2989 2988 2022-06-12T13:04:45Z Neil 2 /* Creating archives */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I use [https://borgbackup.readthedocs.io/en/stable/ Borg] for taking backups of my two machines every night. It's controlled by a SystemD timer. The deduplication of backups allows many backups to be made in not much space. Borg allows easy recovery by allowing archives to be mounted as a drive. The desktop and server machine each have two repositories. Each machine creates a backup locally and one on the other machine. I do it this way so there are no clashes with concurrent backups writing to the same repository from different machines. == Create users == Backups are safer if they use non-privileged users to do the backups. I create these users for the OS and databases. === OS=== On each machine, create <code>backup</code> users. These will be used as the target of backups to remote machines. Give these users login shells and home directories. For each user, create an SSH key and transfer it to the other machine with <code>ssh-copy-id</code>. Once you've set up that both <code>root</code> and <code>backup</code> on each machine can SSH into <code>backup</code> on the other machine, you can disable the passwords for each <code>backup</code> user. === MySQL === For instance, create the <code>backup</code> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. === Postgres === For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Creating archives == As the <code>backup</code> user, create two repositories on each machine, one for each machine being backed up. Remember to check and write down the repository encryption keys! root@desktop:~# mkdir -p /backup/from_desktop root@desktop:~# mkdir -p /backup/from_server root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_desktop root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_server Do the same on the server, creating the two repositories. On each machine, create the file <code>/etc/borg/backup_patterns</code> that defines what files are backed up. This allows the same files to be backed up to each destination. P sh R /home/ - /home/*/.cache - /home/*/TresoritDrive - /home/*/.kde/share/apps/nepomuk - /home/*/.thunderbird - /home/*/.local/share - /home/*/.rvm - /home/*/.vagrant.d - /home/*/anaconda3 - /home/*/.ghcup - /home/**/machine-learning-datasets R /opt/photos R /opt/music R /opt/sources R /etc R /lib/systemd R /var/www To do the backups from the desktop, create the files <code>/etc/borg/backup_to_desktop</code> and <code>/etc/borg/backup_to_server</code>. Below is the content of <code>/etc/borg/backup_to_desktop</code>. <code>/etc/borg/backup_to_server</code> is identical except for the line at the top which becomes <code>export BORG_REPO=backup@server:/backup/from_desktop</code> and the appropriate passphrase. #!/bin/sh # the repo and its passphrase export BORG_REPO=/backup/from_desktop export BORG_PASSPHRASE='long complex passphrase' # some helpers and error handling: info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; } trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM info "Starting backup" # backup the directories borg create \ --verbose --filter AME \ --list --stats --show-rc \ --one-file-system \ --compression zstd,22 --exclude-caches \ --patterns-from /etc/borg/backup_patterns \ ::'{hostname}-daily-{now}' 2>&1 backup_exit=$? info "Pruning repository" # prune the repo borg prune \ --list \ --prefix '{hostname}-daily-' \ --show-rc \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 24 2>&1 prune_exit=$? # Compact the repo, for when version 1.2 is available # borg compact # compact_exit=$? # use highest exit code as exit code global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit )) # global_exit=$(( compact_exit > global_exit ? compact_exit : global_exit )) if [ ${global_exit} -eq 1 ]; then info "Backup, Prune, or Compact finished with a warning" fi if [ ${global_exit} -gt 1 ]; then info "Backup, Prune, or Compact finished with an error" fi exit ${global_exit} Backing up the server requires an extra step, as I want to create database dumps that are backed up first. That means there are four Bash scripts in <code>/etc/borg</code>: <code>backup_starter</code> coordinates the backup. It calls <code>create_db_backups</code> to dump the databases, then the two <code>backup_to_server</code> and <code>backup_to_desktop</code> scripts. <code>backup_starter</code> is simple: #!/bin/bash /etc/borg/create_db_backups /etc/borg/backup_to_server & /etc/borg/backup_to_desktop & wait <code>create_db_backups</code> does the database dumps, as well as creating a list of manually-installed packages. #!/bin/bash export DB_BACKUP_DIR=/backup/db_backups rm ${DB_BACKUP_DIR}/* || true ( nice comm -23 \ <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) \ <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) \ > "${DB_BACKUP_DIR}/manually-installed-packages.txt" ) & # Create database backups for db in thisdb thatdb ; do ( nice mysqldump --user='backup' --opt --databases $db > "${DB_BACKUP_DIR}/${db}-dump.sql" ) & done ( pg_dump -h 127.0.0.1 -U backup -Fc dmarc > "${DB_BACKUP_DIR}/dmarc-dump" ) & for wiki in this_wiki that_wiki ; do ( php /usr/share/mediawiki/maintenance/dumpBackup.php --full --quiet \ --conf /var/www/${wiki}/mediawiki/LocalSettings.php \ > "${DB_BACKUP_DIR}/${wiki}.dump.xml" ) & done wait <code>backup_to_server</code> and <code>backup_to_desktop</code> are as above, but include the <code>/backup/db_backups</code> directory in the patterns file. == Services and timers == === SystemD version === The timing of Anacron is controlled by SystemD, as specified in the <code>/etc/systemd/system/timers.target.wants/anacron.timer</code> file. Change the file to have the updated <code>OnCalendar</code> line: [Unit] Description=Trigger anacron every hour [Timer] # OnCalendar=*-*-* 07..23:30 OnCalendar=*-*-* 01:03 RandomizedDelaySec=5m Persistent=true [Install] WantedBy=timers.target Then restart the <code>anacron</code> daemon. root@desktop:~# systemctl restart anacron.service == Borg == : Just notes at the moment. * [https://borgbackup.readthedocs.io/en/stable/quickstart.html# Borg], and [https://practical-admin.com/blog/backups-using-borg/ a guide] and [https://blog.andrewkeech.com/posts/170718_borg.html another guide]. * Create backup users on each machine. Give them login shells, home directories. Initially create passwords. * Create ssh keys for each * ssh-copy-id for backup and root onto each machine. * Check permissions of .ssh and .ssh/authorized_keys * Delete passwords * Systemd scripts in /lib/systemd/system/ * systemctl daemon-reload * systemctl enable borg_backup_to_desktop.timer * systemctl restart timers.target Extracting files, probably easier with <code>borg mount</code> [[Category:Server setup]] 67cc6767f755f5bd87c9705c49a1f24b006a473a 2990 2989 2022-06-12T13:06:10Z Neil 2 /* Creating archives */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I use [https://borgbackup.readthedocs.io/en/stable/ Borg] for taking backups of my two machines every night. It's controlled by a SystemD timer. The deduplication of backups allows many backups to be made in not much space. Borg allows easy recovery by allowing archives to be mounted as a drive. The desktop and server machine each have two repositories. Each machine creates a backup locally and one on the other machine. I do it this way so there are no clashes with concurrent backups writing to the same repository from different machines. == Create users == Backups are safer if they use non-privileged users to do the backups. I create these users for the OS and databases. === OS=== On each machine, create <code>backup</code> users. These will be used as the target of backups to remote machines. Give these users login shells and home directories. For each user, create an SSH key and transfer it to the other machine with <code>ssh-copy-id</code>. Once you've set up that both <code>root</code> and <code>backup</code> on each machine can SSH into <code>backup</code> on the other machine, you can disable the passwords for each <code>backup</code> user. === MySQL === For instance, create the <code>backup</code> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. === Postgres === For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Creating archives == As the <code>backup</code> user, create two repositories on each machine, one for each machine being backed up. Remember to check and write down the repository encryption keys! root@desktop:~# mkdir -p /backup/from_desktop root@desktop:~# mkdir -p /backup/from_server root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_desktop root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_server Do the same on the server, creating the two repositories. On each machine, create the file <code>/etc/borg/backup_patterns</code> that defines what files are backed up. This allows the same files to be backed up to each destination. P sh R /home/ - /home/*/.cache - /home/*/TresoritDrive - /home/*/.kde/share/apps/nepomuk - /home/*/.thunderbird - /home/*/.local/share - /home/*/.rvm - /home/*/.vagrant.d - /home/*/anaconda3 - /home/*/.ghcup - /home/**/machine-learning-datasets R /opt/photos R /opt/music R /opt/sources R /etc R /lib/systemd R /var/www To do the backups from the desktop, create the files <code>/etc/borg/backup_to_desktop</code> and <code>/etc/borg/backup_to_server</code>. Below is the content of <code>/etc/borg/backup_to_desktop</code>. <code>/etc/borg/backup_to_server</code> is identical except for the line at the top which becomes <code>export BORG_REPO=backup@server:/backup/from_desktop</code> and the appropriate passphrase. #!/bin/sh # the repo and its passphrase export BORG_REPO=/backup/from_desktop export BORG_PASSPHRASE='long complex passphrase' # some helpers and error handling: info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; } trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM info "Starting backup" # backup the directories borg create \ --verbose --filter AME \ --list --stats --show-rc \ --one-file-system \ --compression zstd,22 --exclude-caches \ --patterns-from /etc/borg/backup_patterns \ ::'{hostname}-daily-{now}' 2>&1 backup_exit=$? info "Pruning repository" # prune the repo borg prune \ --list \ --prefix '{hostname}-daily-' \ --show-rc \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 24 2>&1 prune_exit=$? # Compact the repo, for when version 1.2 is available # borg compact # compact_exit=$? # use highest exit code as exit code global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit )) # global_exit=$(( compact_exit > global_exit ? compact_exit : global_exit )) if [ ${global_exit} -eq 1 ]; then info "Backup, Prune, or Compact finished with a warning" fi if [ ${global_exit} -gt 1 ]; then info "Backup, Prune, or Compact finished with an error" fi exit ${global_exit} Backing up the server requires an extra step, as I want to create database dumps that are backed up first. That means there are four Bash scripts in <code>/etc/borg</code>: <code>backup_starter</code> coordinates the backup. It calls <code>create_db_backups</code> to dump the databases, then the two <code>backup_to_server</code> and <code>backup_to_desktop</code> scripts. <code>backup_starter</code> is simple: #!/bin/bash /etc/borg/create_db_backups /etc/borg/backup_to_server & /etc/borg/backup_to_desktop & wait <code>create_db_backups</code> does the database dumps, as well as creating a list of manually-installed packages. #!/bin/bash export DB_BACKUP_DIR=/backup/db_backups rm ${DB_BACKUP_DIR}/* || true ( nice comm -23 \ <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) \ <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) \ > "${DB_BACKUP_DIR}/manually-installed-packages.txt" ) & # Create database backups for db in thisdb thatdb ; do ( nice mysqldump --user='backup' --opt --databases $db > "${DB_BACKUP_DIR}/${db}-dump.sql" ) & done ( pg_dump -h 127.0.0.1 -U backup -Fc dmarc > "${DB_BACKUP_DIR}/dmarc-dump" ) & for wiki in this_wiki that_wiki ; do ( php /usr/share/mediawiki/maintenance/dumpBackup.php --full --quiet \ --conf /var/www/${wiki}/mediawiki/LocalSettings.php \ > "${DB_BACKUP_DIR}/${wiki}.dump.xml" ) & done wait <code>backup_to_server</code> and <code>backup_to_desktop</code> are as above, but include the <code>/backup/db_backups</code> directory in the patterns file. All the Bash scripts need to be executable. == Services and timers == === SystemD version === The timing of Anacron is controlled by SystemD, as specified in the <code>/etc/systemd/system/timers.target.wants/anacron.timer</code> file. Change the file to have the updated <code>OnCalendar</code> line: [Unit] Description=Trigger anacron every hour [Timer] # OnCalendar=*-*-* 07..23:30 OnCalendar=*-*-* 01:03 RandomizedDelaySec=5m Persistent=true [Install] WantedBy=timers.target Then restart the <code>anacron</code> daemon. root@desktop:~# systemctl restart anacron.service == Borg == : Just notes at the moment. * [https://borgbackup.readthedocs.io/en/stable/quickstart.html# Borg], and [https://practical-admin.com/blog/backups-using-borg/ a guide] and [https://blog.andrewkeech.com/posts/170718_borg.html another guide]. * Create backup users on each machine. Give them login shells, home directories. Initially create passwords. * Create ssh keys for each * ssh-copy-id for backup and root onto each machine. * Check permissions of .ssh and .ssh/authorized_keys * Delete passwords * Systemd scripts in /lib/systemd/system/ * systemctl daemon-reload * systemctl enable borg_backup_to_desktop.timer * systemctl restart timers.target Extracting files, probably easier with <code>borg mount</code> [[Category:Server setup]] 459942827189cf4ff637a900afb08e3c34058830 2991 2990 2022-06-12T13:12:30Z Neil 2 /* Services and timers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I use [https://borgbackup.readthedocs.io/en/stable/ Borg] for taking backups of my two machines every night. It's controlled by a SystemD timer. The deduplication of backups allows many backups to be made in not much space. Borg allows easy recovery by allowing archives to be mounted as a drive. The desktop and server machine each have two repositories. Each machine creates a backup locally and one on the other machine. I do it this way so there are no clashes with concurrent backups writing to the same repository from different machines. == Create users == Backups are safer if they use non-privileged users to do the backups. I create these users for the OS and databases. === OS=== On each machine, create <code>backup</code> users. These will be used as the target of backups to remote machines. Give these users login shells and home directories. For each user, create an SSH key and transfer it to the other machine with <code>ssh-copy-id</code>. Once you've set up that both <code>root</code> and <code>backup</code> on each machine can SSH into <code>backup</code> on the other machine, you can disable the passwords for each <code>backup</code> user. === MySQL === For instance, create the <code>backup</code> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. === Postgres === For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Creating archives == As the <code>backup</code> user, create two repositories on each machine, one for each machine being backed up. Remember to check and write down the repository encryption keys! root@desktop:~# mkdir -p /backup/from_desktop root@desktop:~# mkdir -p /backup/from_server root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_desktop root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_server Do the same on the server, creating the two repositories. On each machine, create the file <code>/etc/borg/backup_patterns</code> that defines what files are backed up. This allows the same files to be backed up to each destination. P sh R /home/ - /home/*/.cache - /home/*/TresoritDrive - /home/*/.kde/share/apps/nepomuk - /home/*/.thunderbird - /home/*/.local/share - /home/*/.rvm - /home/*/.vagrant.d - /home/*/anaconda3 - /home/*/.ghcup - /home/**/machine-learning-datasets R /opt/photos R /opt/music R /opt/sources R /etc R /lib/systemd R /var/www To do the backups from the desktop, create the files <code>/etc/borg/backup_to_desktop</code> and <code>/etc/borg/backup_to_server</code>. Below is the content of <code>/etc/borg/backup_to_desktop</code>. <code>/etc/borg/backup_to_server</code> is identical except for the line at the top which becomes <code>export BORG_REPO=backup@server:/backup/from_desktop</code> and the appropriate passphrase. #!/bin/sh # the repo and its passphrase export BORG_REPO=/backup/from_desktop export BORG_PASSPHRASE='long complex passphrase' # some helpers and error handling: info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; } trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM info "Starting backup" # backup the directories borg create \ --verbose --filter AME \ --list --stats --show-rc \ --one-file-system \ --compression zstd,22 --exclude-caches \ --patterns-from /etc/borg/backup_patterns \ ::'{hostname}-daily-{now}' 2>&1 backup_exit=$? info "Pruning repository" # prune the repo borg prune \ --list \ --prefix '{hostname}-daily-' \ --show-rc \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 24 2>&1 prune_exit=$? # Compact the repo, for when version 1.2 is available # borg compact # compact_exit=$? # use highest exit code as exit code global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit )) # global_exit=$(( compact_exit > global_exit ? compact_exit : global_exit )) if [ ${global_exit} -eq 1 ]; then info "Backup, Prune, or Compact finished with a warning" fi if [ ${global_exit} -gt 1 ]; then info "Backup, Prune, or Compact finished with an error" fi exit ${global_exit} Backing up the server requires an extra step, as I want to create database dumps that are backed up first. That means there are four Bash scripts in <code>/etc/borg</code>: <code>backup_starter</code> coordinates the backup. It calls <code>create_db_backups</code> to dump the databases, then the two <code>backup_to_server</code> and <code>backup_to_desktop</code> scripts. <code>backup_starter</code> is simple: #!/bin/bash /etc/borg/create_db_backups /etc/borg/backup_to_server & /etc/borg/backup_to_desktop & wait <code>create_db_backups</code> does the database dumps, as well as creating a list of manually-installed packages. #!/bin/bash export DB_BACKUP_DIR=/backup/db_backups rm ${DB_BACKUP_DIR}/* || true ( nice comm -23 \ <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) \ <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) \ > "${DB_BACKUP_DIR}/manually-installed-packages.txt" ) & # Create database backups for db in thisdb thatdb ; do ( nice mysqldump --user='backup' --opt --databases $db > "${DB_BACKUP_DIR}/${db}-dump.sql" ) & done ( pg_dump -h 127.0.0.1 -U backup -Fc dmarc > "${DB_BACKUP_DIR}/dmarc-dump" ) & for wiki in this_wiki that_wiki ; do ( php /usr/share/mediawiki/maintenance/dumpBackup.php --full --quiet \ --conf /var/www/${wiki}/mediawiki/LocalSettings.php \ > "${DB_BACKUP_DIR}/${wiki}.dump.xml" ) & done wait <code>backup_to_server</code> and <code>backup_to_desktop</code> are as above, but include the <code>/backup/db_backups</code> directory in the patterns file. All the Bash scripts need to be executable. == Services and timers == The backups are run by SystemD services and timers. On the server, create <code>/lib/systemd/system/borg_backup.service</code> and <code>/lib/systemd/system/borg_backup.timer</code>. * <code>/lib/systemd/system/borg_backup.service</code> [Unit] Description=Borg Backup to server and desktop [Service] Type=simple Nice=19 IOSchedulingClass=2 IOSchedulingPriority=7 ExecStart=/etc/borg/backup_starter # User=backup # Group=backup * <code>/lib/systemd/system/borg_backup.timer</code> [Unit] Description=Borg Backup Timer [Timer] WakeSystem=false OnCalendar=*-*-* 1:00:00 RandomizedDelaySec=10min [Install] WantedBy=timers.target The desktop machine has two service and timer files, one for each backup script: the backup to the desktop and backup to the server run as separate services. == Borg == : Just notes at the moment. * [https://borgbackup.readthedocs.io/en/stable/quickstart.html# Borg], and [https://practical-admin.com/blog/backups-using-borg/ a guide] and [https://blog.andrewkeech.com/posts/170718_borg.html another guide]. * Create backup users on each machine. Give them login shells, home directories. Initially create passwords. * Create ssh keys for each * ssh-copy-id for backup and root onto each machine. * Check permissions of .ssh and .ssh/authorized_keys * Delete passwords * Systemd scripts in /lib/systemd/system/ * systemctl daemon-reload * systemctl enable borg_backup_to_desktop.timer * systemctl restart timers.target Extracting files, probably easier with <code>borg mount</code> [[Category:Server setup]] 18681ac2f66c01174a47ab5812324fb26bd61878 2992 2991 2022-06-12T13:14:18Z Neil 2 /* Services and timers */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I use [https://borgbackup.readthedocs.io/en/stable/ Borg] for taking backups of my two machines every night. It's controlled by a SystemD timer. The deduplication of backups allows many backups to be made in not much space. Borg allows easy recovery by allowing archives to be mounted as a drive. The desktop and server machine each have two repositories. Each machine creates a backup locally and one on the other machine. I do it this way so there are no clashes with concurrent backups writing to the same repository from different machines. == Create users == Backups are safer if they use non-privileged users to do the backups. I create these users for the OS and databases. === OS=== On each machine, create <code>backup</code> users. These will be used as the target of backups to remote machines. Give these users login shells and home directories. For each user, create an SSH key and transfer it to the other machine with <code>ssh-copy-id</code>. Once you've set up that both <code>root</code> and <code>backup</code> on each machine can SSH into <code>backup</code> on the other machine, you can disable the passwords for each <code>backup</code> user. === MySQL === For instance, create the <code>backup</code> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. === Postgres === For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Creating archives == As the <code>backup</code> user, create two repositories on each machine, one for each machine being backed up. Remember to check and write down the repository encryption keys! root@desktop:~# mkdir -p /backup/from_desktop root@desktop:~# mkdir -p /backup/from_server root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_desktop root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_server Do the same on the server, creating the two repositories. On each machine, create the file <code>/etc/borg/backup_patterns</code> that defines what files are backed up. This allows the same files to be backed up to each destination. P sh R /home/ - /home/*/.cache - /home/*/TresoritDrive - /home/*/.kde/share/apps/nepomuk - /home/*/.thunderbird - /home/*/.local/share - /home/*/.rvm - /home/*/.vagrant.d - /home/*/anaconda3 - /home/*/.ghcup - /home/**/machine-learning-datasets R /opt/photos R /opt/music R /opt/sources R /etc R /lib/systemd R /var/www To do the backups from the desktop, create the files <code>/etc/borg/backup_to_desktop</code> and <code>/etc/borg/backup_to_server</code>. Below is the content of <code>/etc/borg/backup_to_desktop</code>. <code>/etc/borg/backup_to_server</code> is identical except for the line at the top which becomes <code>export BORG_REPO=backup@server:/backup/from_desktop</code> and the appropriate passphrase. #!/bin/sh # the repo and its passphrase export BORG_REPO=/backup/from_desktop export BORG_PASSPHRASE='long complex passphrase' # some helpers and error handling: info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; } trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM info "Starting backup" # backup the directories borg create \ --verbose --filter AME \ --list --stats --show-rc \ --one-file-system \ --compression zstd,22 --exclude-caches \ --patterns-from /etc/borg/backup_patterns \ ::'{hostname}-daily-{now}' 2>&1 backup_exit=$? info "Pruning repository" # prune the repo borg prune \ --list \ --prefix '{hostname}-daily-' \ --show-rc \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 24 2>&1 prune_exit=$? # Compact the repo, for when version 1.2 is available # borg compact # compact_exit=$? # use highest exit code as exit code global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit )) # global_exit=$(( compact_exit > global_exit ? compact_exit : global_exit )) if [ ${global_exit} -eq 1 ]; then info "Backup, Prune, or Compact finished with a warning" fi if [ ${global_exit} -gt 1 ]; then info "Backup, Prune, or Compact finished with an error" fi exit ${global_exit} Backing up the server requires an extra step, as I want to create database dumps that are backed up first. That means there are four Bash scripts in <code>/etc/borg</code>: <code>backup_starter</code> coordinates the backup. It calls <code>create_db_backups</code> to dump the databases, then the two <code>backup_to_server</code> and <code>backup_to_desktop</code> scripts. <code>backup_starter</code> is simple: #!/bin/bash /etc/borg/create_db_backups /etc/borg/backup_to_server & /etc/borg/backup_to_desktop & wait <code>create_db_backups</code> does the database dumps, as well as creating a list of manually-installed packages. #!/bin/bash export DB_BACKUP_DIR=/backup/db_backups rm ${DB_BACKUP_DIR}/* || true ( nice comm -23 \ <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) \ <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) \ > "${DB_BACKUP_DIR}/manually-installed-packages.txt" ) & # Create database backups for db in thisdb thatdb ; do ( nice mysqldump --user='backup' --opt --databases $db > "${DB_BACKUP_DIR}/${db}-dump.sql" ) & done ( pg_dump -h 127.0.0.1 -U backup -Fc dmarc > "${DB_BACKUP_DIR}/dmarc-dump" ) & for wiki in this_wiki that_wiki ; do ( php /usr/share/mediawiki/maintenance/dumpBackup.php --full --quiet \ --conf /var/www/${wiki}/mediawiki/LocalSettings.php \ > "${DB_BACKUP_DIR}/${wiki}.dump.xml" ) & done wait <code>backup_to_server</code> and <code>backup_to_desktop</code> are as above, but include the <code>/backup/db_backups</code> directory in the patterns file. All the Bash scripts need to be executable. == Services and timers == The backups are run by SystemD services and timers. On the server, create <code>/lib/systemd/system/borg_backup.service</code> and <code>/lib/systemd/system/borg_backup.timer</code>. * <code>/lib/systemd/system/borg_backup.service</code> [Unit] Description=Borg Backup to server and desktop [Service] Type=simple Nice=19 IOSchedulingClass=2 IOSchedulingPriority=7 ExecStart=/etc/borg/backup_starter # User=backup # Group=backup * <code>/lib/systemd/system/borg_backup.timer</code> [Unit] Description=Borg Backup Timer [Timer] WakeSystem=false OnCalendar=*-*-* 1:00:00 RandomizedDelaySec=10min [Install] WantedBy=timers.target The desktop machine has two service and timer files, one for each backup script: the backup to the desktop and backup to the server run as separate services. Once you've created the service and timer files, tell SystemD about them and start the timers. root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl enable borg_backup_to_desktop.timer root@desktop:~# systemctl restart timers.target == Borg == : Just notes at the moment. * [https://borgbackup.readthedocs.io/en/stable/quickstart.html# Borg], and [https://practical-admin.com/blog/backups-using-borg/ a guide] and [https://blog.andrewkeech.com/posts/170718_borg.html another guide]. * Create backup users on each machine. Give them login shells, home directories. Initially create passwords. * Create ssh keys for each * ssh-copy-id for backup and root onto each machine. * Check permissions of .ssh and .ssh/authorized_keys * Delete passwords * Systemd scripts in /lib/systemd/system/ * systemctl daemon-reload * systemctl enable borg_backup_to_desktop.timer * systemctl restart timers.target Extracting files, probably easier with <code>borg mount</code> [[Category:Server setup]] 278c8dfc0044b142b1aafd611d83d4e2aecab69a 2993 2992 2022-06-12T13:19:17Z Neil 2 /* Borg */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Log viewer setup|Log stats]] |align="center" width="50%"| [[Discourse setup|Discourse]] |} I use [https://borgbackup.readthedocs.io/en/stable/ Borg] for taking backups of my two machines every night. It's controlled by a SystemD timer. The deduplication of backups allows many backups to be made in not much space. Borg allows easy recovery by allowing archives to be mounted as a drive. The desktop and server machine each have two repositories. Each machine creates a backup locally and one on the other machine. I do it this way so there are no clashes with concurrent backups writing to the same repository from different machines. == Create users == Backups are safer if they use non-privileged users to do the backups. I create these users for the OS and databases. === OS=== On each machine, create <code>backup</code> users. These will be used as the target of backups to remote machines. Give these users login shells and home directories. For each user, create an SSH key and transfer it to the other machine with <code>ssh-copy-id</code>. Once you've set up that both <code>root</code> and <code>backup</code> on each machine can SSH into <code>backup</code> on the other machine, you can disable the passwords for each <code>backup</code> user. === MySQL === For instance, create the <code>backup</code> user in MySQL. This user has read-only access to all databases, and does not require a password. Create the user by entering the following commands: root@desktop:~# mysql -u 'root' -p mysql> grant select, lock tables, process on *.* to 'backup'@'localhost'; mysql> quit; This automatically creates the user, with no password, and gives them permissions to read all MySQL databases. === Postgres === For Postgres, it's a bit more complicated. Create the user, connect to the database to back up, and grant permissions. root@desktop:~# sudo -u postgres psql postgres=# create user backup with login encrypted password 'mypassword'; CREATE ROLE postgres=# \c dmarc psql (9.3.13, server 9.1.18) You are now connected to database "dmarc" as user "postgres". postgres=# grant connect on database dmarc to backup; GRANT postgres=# grant select on all tables in schema public to backup; GRANT postgres=# grant select on all sequences in schema public to backup; GRANT Then create the file <code>/root/.pgpass</code> with contents # hostname:port:database:username:password *:*:*:backup:mypassword : (giving the password you've used) and give it the correct permissions: root@desktop:~# chmod 0600 ~/.pgpass == Creating archives == As the <code>backup</code> user, create two repositories on each machine, one for each machine being backed up. Remember to check and write down the repository encryption keys! root@desktop:~# mkdir -p /backup/from_desktop root@desktop:~# mkdir -p /backup/from_server root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_desktop root@desktop:~# sudo -u backup borg init --encryption=repokey /backup/from_server Do the same on the server, creating the two repositories. On each machine, create the file <code>/etc/borg/backup_patterns</code> that defines what files are backed up. This allows the same files to be backed up to each destination. P sh R /home/ - /home/*/.cache - /home/*/TresoritDrive - /home/*/.kde/share/apps/nepomuk - /home/*/.thunderbird - /home/*/.local/share - /home/*/.rvm - /home/*/.vagrant.d - /home/*/anaconda3 - /home/*/.ghcup - /home/**/machine-learning-datasets R /opt/photos R /opt/music R /opt/sources R /etc R /lib/systemd R /var/www To do the backups from the desktop, create the files <code>/etc/borg/backup_to_desktop</code> and <code>/etc/borg/backup_to_server</code>. Below is the content of <code>/etc/borg/backup_to_desktop</code>. <code>/etc/borg/backup_to_server</code> is identical except for the line at the top which becomes <code>export BORG_REPO=backup@server:/backup/from_desktop</code> and the appropriate passphrase. #!/bin/sh # the repo and its passphrase export BORG_REPO=/backup/from_desktop export BORG_PASSPHRASE='long complex passphrase' # some helpers and error handling: info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; } trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM info "Starting backup" # backup the directories borg create \ --verbose --filter AME \ --list --stats --show-rc \ --one-file-system \ --compression zstd,22 --exclude-caches \ --patterns-from /etc/borg/backup_patterns \ ::'{hostname}-daily-{now}' 2>&1 backup_exit=$? info "Pruning repository" # prune the repo borg prune \ --list \ --prefix '{hostname}-daily-' \ --show-rc \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 24 2>&1 prune_exit=$? # Compact the repo, for when version 1.2 is available # borg compact # compact_exit=$? # use highest exit code as exit code global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit )) # global_exit=$(( compact_exit > global_exit ? compact_exit : global_exit )) if [ ${global_exit} -eq 1 ]; then info "Backup, Prune, or Compact finished with a warning" fi if [ ${global_exit} -gt 1 ]; then info "Backup, Prune, or Compact finished with an error" fi exit ${global_exit} Backing up the server requires an extra step, as I want to create database dumps that are backed up first. That means there are four Bash scripts in <code>/etc/borg</code>: <code>backup_starter</code> coordinates the backup. It calls <code>create_db_backups</code> to dump the databases, then the two <code>backup_to_server</code> and <code>backup_to_desktop</code> scripts. <code>backup_starter</code> is simple: #!/bin/bash /etc/borg/create_db_backups /etc/borg/backup_to_server & /etc/borg/backup_to_desktop & wait <code>create_db_backups</code> does the database dumps, as well as creating a list of manually-installed packages. #!/bin/bash export DB_BACKUP_DIR=/backup/db_backups rm ${DB_BACKUP_DIR}/* || true ( nice comm -23 \ <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) \ <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) \ > "${DB_BACKUP_DIR}/manually-installed-packages.txt" ) & # Create database backups for db in thisdb thatdb ; do ( nice mysqldump --user='backup' --opt --databases $db > "${DB_BACKUP_DIR}/${db}-dump.sql" ) & done ( pg_dump -h 127.0.0.1 -U backup -Fc dmarc > "${DB_BACKUP_DIR}/dmarc-dump" ) & for wiki in this_wiki that_wiki ; do ( php /usr/share/mediawiki/maintenance/dumpBackup.php --full --quiet \ --conf /var/www/${wiki}/mediawiki/LocalSettings.php \ > "${DB_BACKUP_DIR}/${wiki}.dump.xml" ) & done wait <code>backup_to_server</code> and <code>backup_to_desktop</code> are as above, but include the <code>/backup/db_backups</code> directory in the patterns file. All the Bash scripts need to be executable. == Services and timers == The backups are run by SystemD services and timers. On the server, create <code>/lib/systemd/system/borg_backup.service</code> and <code>/lib/systemd/system/borg_backup.timer</code>. * <code>/lib/systemd/system/borg_backup.service</code> [Unit] Description=Borg Backup to server and desktop [Service] Type=simple Nice=19 IOSchedulingClass=2 IOSchedulingPriority=7 ExecStart=/etc/borg/backup_starter # User=backup # Group=backup * <code>/lib/systemd/system/borg_backup.timer</code> [Unit] Description=Borg Backup Timer [Timer] WakeSystem=false OnCalendar=*-*-* 1:00:00 RandomizedDelaySec=10min [Install] WantedBy=timers.target The desktop machine has two service and timer files, one for each backup script: the backup to the desktop and backup to the server run as separate services. Once you've created the service and timer files, tell SystemD about them and start the timers. root@desktop:~# systemctl daemon-reload root@desktop:~# systemctl enable borg_backup_to_desktop.timer root@desktop:~# systemctl restart timers.target == Restoring files == The easiest way is to use <code>borg list</code> to view the backups in each repository, and <code>borg mount</code> to mount the repository as a filesystem. You can then extract the files you want fairly easily. == See also == * [https://borgbackup.readthedocs.io/en/stable/index.html Borg documentation] * [https://practical-admin.com/blog/backups-using-borg/ Andrew Sullivan's guide] to Borg * [https://blog.andrewkeech.com/posts/170718_borg.html Andrew Keech's guide] to Borg * [https://torsion.org/borgmatic/ Borgmatic] may be an interesting tool, if I want something more configurable. [[Category:Server setup]] 94a607be24add673d3d09e5bc0ecef0e170e7e97 0 1553 2995 2068 2022-07-22T18:31:09Z Neil 2 IPv6 notes wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| None |align="center" width="50%"| [[OS installation]] |} The router needs a bit of setting up to allow all the server services to be available. This should be done in stages, as things are installed on the server. This document collects them all together for convenience. == Get it working == The first step is to simply get the router working, so that it connects to your ISP and can handle satellite machines connecting via DHCP. == DHCP ranges == In order for the machines with [[Hostname and IP setup|fixed IP numbers]] to avoid stomping all over the router's DHCP server, you need to ensure that some addresses on the local network are not available for allocation by the router's DHCP server. Configure the router so that addresses from 192.168.1.250 to 192.168.1.254 are outside the DHCP pool. == SLAAC and DHCPv6 == To use IPv6 devices, ensure the router knows how to allocate addresses to them. It should have a /64 network and be able to support IP reservation by SLAAC. It should also have a DHCPv6 server operating in a small space; mine uses a /112 address space for DHCPv6. == DNS rerouting == By default, the router uses the ISP's DNS servers to find addresses for its DHCP-connected machines. Unfortunately, doing this means that these machines won't be able to find the addresses of the various machines on the LAN. Once the [[DNS server setup|DNS servers]] are set up, change the router's settings so that it no longer gets DNS server information from the ISP, but uses the nameservers you specify: specify the ones you've just set up. == Port forwarding == To make the various services you've set up available to the outside world, you'll need to pass requests from the outside world into the server. The router should have a 'port forwarding' facility or similar, where incoming traffic to the router on a particular port is passed to a specified IP number and port on the LAN. Once you've set up a service, you can open the port for it. The ports you'll need to open are: * Port 25 for [[Postfix server setup|SMTP mail transport]] * Port 80 for [[Web server setup|serving Web pages]] * Port 443 for secure web pages, such as used by [[Webmail setup|webmail]] You may want to consider opening port 22 (SSH) which will allow secure remote logins to a machine. If you do, you'll probably need to protect it against opportunistic login attempts by passersby. See the [[Firewall_setup#Protect_the_SSH_port|discussion on the firewall page]] for details. For IPv6 devices, you don't need to forward ports but you will need to expose the global IP numbers of the servers. [[Category:Server setup]] 26aedb0df0a383f0f1d0c1636252afff420bfcb5 0 1 3003 2957 2022-11-07T12:31:20Z Neil 2 wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == <div style="float: right;"> {| {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://wheretofind.me/@NeilNjae NeilNjae] on WhereToFindMe * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://dice.camp/web/@neilnjae @NeilNjae@dice.camp] on Mastodon <!-- * [https://ello.co/neilnjae @neilnjae] on Ello --> * [https://keybase.io/neilnjae NeilNjae] on Keybase.io <!-- * [https://pluspora.com/people/e1c57380ade5013646c5005056268def NeilNjae] on Plusporia --> <!-- * [https://mewe.com/profile/5bbbb775a5f4e527d1b3d9e5 NeilNjae] on MeWe --> * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] * Send me BitCoins at [https://blockchain.info/address/1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV 1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV] |} </div> === [[:Category:RPGs|RPGs]] === Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. === [[server setup|Servers and sysadmin]] === I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). === Other stuff === I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://tresorit.com/ Tresorit] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. fba6e1553cd62b46cf4d13329ea0e9ad97e82544 3004 3003 2022-11-12T10:37:51Z Neil 2 Added Mastodon link attribute wikitext text/x-wiki <big>'''Welcome to the Smith Family Homepage'''</big> __NOTOC__ == Neil == <div style="float: right;"> {| {{Prettytable}} ! {{Hl3}}|'''Elsewhere, I am...''' |- | * [https://wheretofind.me/@NeilNjae NeilNjae] on WhereToFindMe * [https://twitter.com/#!/neilnjae @neilnjae] on Twitter * [https://www.facebook.com/profile.php?id=659289774 Neil Smith] on Facebook * [https://github.com/NeilNjae NeilNjae] on Github * [https://dice.camp/web/@neilnjae @NeilNjae@dice.camp((me))] on Mastodon <!-- * [https://ello.co/neilnjae @neilnjae] on Ello --> * [https://keybase.io/neilnjae NeilNjae] on Keybase.io <!-- * [https://pluspora.com/people/e1c57380ade5013646c5005056268def NeilNjae] on Plusporia --> <!-- * [https://mewe.com/profile/5bbbb775a5f4e527d1b3d9e5 NeilNjae] on MeWe --> * My [[Neil-public-key.asc|GnuPG key]] (fingerprint 3B3E06EA), :also on [https://keyserver.pgp.com/ PGP Global Directory] * Send me BitCoins at [https://blockchain.info/address/1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV 1Mgx2FU92AXbd43i7838agU8i1gbiSgXqV] |} </div> === [[:Category:RPGs|RPGs]] === Most of my pages are about [[:Category:RPGs|role-playing games]] (RPGs), which I do at the [http://www.mk-rpg.org.uk Milton Keynes Roleplaying Games club]. I have some stuff on the [[:Category:HeroQuest|HeroQuest]] game and the fictional world of [[:Category:Glorantha|Glorantha]] as well as some pages on the [[:Category:Blue Planet|Blue Planet]] game. I've also mucked around with some [[:Category:Name generation|name generation]] programs. === [[server setup|Servers and sysadmin]] === I've also written some notes on [[server setup|setting up my various server machines]] (mainly for my own reference) and [[:Category:Rails|building and installing Rails applications]]. [https://github.com/NeilNjae I have some stuff on Github]. I also have a few [[downloadable programs]] (nothing exciting). === Other stuff === I did use SpiderOak, but found that [[SpiderOak is unsafe]]. [https://tresorit.com/ Tresorit] seems to be working well for me as a secure online backup service. I do [[:Category:Sous vide|sous vide cooking]] (cooking for long periods in a temperature-controlled water bath) and have some notes on my cheap, easy, DIY [[sous vide setup mk 1|simple sous vide setup]]. And finally, I go to the [http://www.jugglingdb.com/clubs/index.php?record=76 Jugglers Anonymous] juggling club, where I drop balls, clubs, and [http://en.wikipedia.org/wiki/Devil_stick devil sticks]. The [http://jugglewiki.org/wiki/Main_Page JuggleWiki] and [http://kingscascade.com/JugglingTricks.html Kings Cascade] have lots of animations of tricks I can't do. == Jenni == I do [[Chemistry tutoring|A-level Chemistry tutoring]]. == [http://www.andrewls.me.uk Andrew] == Andrew has [http://www.andrewls.me.uk his own website]. Go there for his pages. == [http://www.eloisejs.me.uk Eloise] == Eloise has [http://www.eloisejs.me.uk her own website]. Go there for her pages. 492458ed2ee8e755b4ee63e2d3f930e242a2de1d 0 1534 3005 2933 2022-11-12T10:46:19Z Neil 2 /* Tweaking Mediawiki */ wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki SetEnv MW_INSTALL_PATH /var/www/site.domain.tld/mediawiki # Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so need to set the MW_INSTALL_PATH explicitly <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(The setting of <code>MW_INSTALL_PATH</code> is taken from the [https://phabricator.wikimedia.org/T203061#4551077 phabricator discussion].) * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/sitemap/ RewriteCond %{REQUEST_URI} !^/.well-known/ RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ %{DOCUMENT_ROOT}/mediawiki/index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the settings around <code>$wgEnableUploads</code> to be: ## To enable image uploads, make sure the 'images' directory ## is writable, then set this to true: $wgEnableUploads = true; $wgUseImageMagick = true; $wgImageMagickConvertCommand = "/usr/bin/convert"; $wgUploadDirectory = "/var/www/domain.tld/mediawiki/images"; $wgUploadPath = "{$wgScriptPath}/images"; ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; $wgGenerateThumbnailOnParse = true; Yes, again, there need to be explicit absolute paths on the server's filesystem. === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Enable the various rendering modes # $wgMathValidModes[] = 'mathml'; $wgMathValidModes[] = 'latex'; // Set the default rendering option # $wgDefaultUserOptions['math'] = 'mathml'; # $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; wgDefaultUserOptions['math'] = 'latex'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). === Link attributes === The [https://www.mediawiki.org/wiki/Extension:Link_Attributes Link Attributes] extension allows some attributes to be added to links, such as the <code>me</code> attribute needed to verify Mastodon links. Installation is simple, just following the instructions. Download and extract the code, then insert the line wfLoadExtension( 'Link_Attributes' ); into <code>LocalSettings.php</code>. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 3e6898f84d202a89a129d397add49d70050c4077 3009 3005 2023-11-30T13:35:37Z Neil 2 Added section on changing upload file limits wikitext text/x-wiki {|align="right" style="width:22em; border:1px solid #aaa; background-color: #f9f9f9; padding: 5px; margin-top: 1em; clear: both;" |colspan="2" align="center"|'''[[Server setup]]''' |- |align="center" width="50%"| &larr; Previous |align="center" width="50%"| Next &rarr; |- |align="center" width="50%"| [[Web server setup|Web server]] |align="center" width="50%"| [[Webmail setup|Webmail]] |} Many of the virtual web hosts on the server will run [http://www.mediawiki.org/wiki/MediaWiki Mediawiki] instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed. == Install the packages == Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts root@server:~# aptitude update root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql Ubuntu doesn't seem to inlcude the ''apc'' PHP cache, which has a great effect on Mediawiki performance. Instead, enable the ''apcu'' cache: root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php == Creating the farm == The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance. * Create the file <code>/var/www/mediawiki-symlinks</code> to create all the symlinks for you; make the file executable. #!/bin/bash for f in /usr/share/mediawiki/api.php \ /usr/share/mediawiki/autoload.php \ /var/lib/mediawiki/extensions/ \ /usr/share/mediawiki/img_auth.php \ /usr/share/mediawiki/includes/ \ /usr/share/mediawiki/index.php \ /usr/share/mediawiki/languages/ \ /usr/share/mediawiki/load.php \ /usr/share/mediawiki/maintenance/ \ /usr/share/mediawiki/mw-config \ /usr/share/mediawiki/opensearch_desc.php \ /usr/share/mediawiki/profileinfo.php \ /usr/share/mediawiki/resources \ /usr/share/mediawiki/serialized \ /usr/share/mediawiki/skins \ /usr/share/mediawiki/thumb_handler.php \ /usr/share/mediawiki/thumb.php \ /usr/share/mediawiki/vendor \ /usr/share/mediawiki/wiki.phtml do ln -s $f done cp -pr /var/lib/mediawiki/cache . cp -pr /var/lib/mediawiki/config . cp -pr /var/lib/mediawiki/images . * For each wiki, create a directory where the wiki's files will sit, and add the copies: root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki root@server:~# cd /var/www/site.domain.tld/mediawiki root@server:~# /var/www/mediawiki-symlinks * Each site will also need it's own directory for creating temporary files. This need not be under the web root. root@server:~# mkdir -p /var/www/site.domain.tld/images/temp root@server:~# mkdir /var/www/site.domain.tld/images/thumb root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp * If you're updating or transferring a site from another host, set up the database and users first. * Modify the Apache configuration file for this site, <code>/etc/apache2/sites-available/site.domain.tld</code>: <VirtualHost *:80> ServerAdmin webmaster@localhost DocumentRoot /var/www/site.domain.tld ServerName site.domain.tld ServerAlias domain.tld Alias /mediawiki /var/www/site.domain.tld/mediawiki SetEnv MW_INSTALL_PATH /var/www/site.domain.tld/mediawiki # Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so need to set the MW_INSTALL_PATH explicitly <Directory /var/www/site.domain.tld/mediawiki/> Options +FollowSymLinks AllowOverride All order allow,deny allow from all </Directory> # some directories must be protected <Directory /var/www/site.domain.tld/mediawiki/config> Options -FollowSymLinks AllowOverride None </Directory> <Directory /var/www/site.domain.tld/mediawiki/images> Options -FollowSymLinks AllowOverride None </Directory> # Rewrite rule will go here: see below ErrorLog /var/log/apache2/error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/access.log combined ServerSignature On </VirtualHost> :(The setting of <code>MW_INSTALL_PATH</code> is taken from the [https://phabricator.wikimedia.org/T203061#4551077 phabricator discussion].) * Reload the Apache config: root@server:~# a2ensite site.domain.tld root@server:~# systemctl reload apache2.service * Now got to <code><nowiki>http://site.domain.tld/mediawiki/</nowiki></code> and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name. * The setup process will finish by downloading a copy of the <code>LocalSettings.php</code> file. Copy it into into the mediawiki root at <code>/var/www/domain.tld/mediawiki</code>: * Add these lines to <code>LocalSettings.php</code>: ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; :Yes, this needs to be an absolute path on the webserver's filesystem. That should be everything needed to get the wiki working. == Rewriting paths == Now to get rid of the <code>/mediawiki/</code> portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine. * Enable the rewriting module root@server:~# a2enmod rewrite * Modify the <code>/etc/apache2/sites-available/site.domain.tld</code> file, in the place specified above. RewriteEngine on RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/ RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php RewriteCond %{REQUEST_URI} !^(load|api).php RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html RewriteCond %{REQUEST_URI} !^/favicon.ico RewriteCond %{REQUEST_URI} !^/robots.txt RewriteCond %{REQUEST_URI} !^/sitemap/ RewriteCond %{REQUEST_URI} !^/.well-known/ RewriteCond %{REQUEST_URI} !^/mediawiki/ # Rewrite <nowiki>http://wiki.domain.tld/article</nowiki> properly, this is the main rule RewriteRule ^/(.*)$ %{DOCUMENT_ROOT}/mediawiki/index.php/?title=$1 [L,QSA] * Reload Apache's confiuration root@server:~# /etc/init.d/apache2 reload * Modify <code>LocalSettings.php</code>, just under the <code>$wgScriptPath</code>, by adding the settings below: $wgScriptPath = "/mediawiki"; ## Added for short names $wgScriptExtension = ".php"; $wgArticlePath = "/$1"; $wgUsePathInfo = false; You should now be able to go to <code><nowiki>http://site.domain.tld/Main_Page</nowiki></code> and see the main page of the wiki. == Tweaking Mediawiki == There are still a few things that need doing in Mediawiki to smooth off some rough edges. === Restricting editing === To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the <tt>LocalSettings.php</tt> file: # Prevent vandalism of the wiki # Prevent new user registrations except by sysops $wgGroupPermissions['*']['createaccount'] = false; # Disable anonymous editing $wgGroupPermissions['*']['edit'] = false; === Restricting viewing === If you want to restrict access to the wiki, add the following to <tt>LocalSettings.php</tt> to ensure that people can only access the wiki's main page. # Disable anonymous browsing $wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer"); $wgGroupPermissions['*']['read'] = false; === Robots file and favicon === Simply put these files in the document root, <tt>/var/www/site.domain.tld</tt>. === Enabling uploads === In <code>LocalSettings.php</code>, modify the settings around <code>$wgEnableUploads</code> to be: ## To enable image uploads, make sure the 'images' directory ## is writable, then set this to true: $wgEnableUploads = true; $wgUseImageMagick = true; $wgImageMagickConvertCommand = "/usr/bin/convert"; $wgUploadDirectory = "/var/www/domain.tld/mediawiki/images"; $wgUploadPath = "{$wgScriptPath}/images"; ## The file path for temporary files. Was /tmp $wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp"; $wgGenerateThumbnailOnParse = true; Yes, again, there need to be explicit absolute paths on the server's filesystem. === Extending uploadable files === Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in <code>LocalSettings.php</code> ## Allow more file types to be uploaded $wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) ); You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in <tt>/usr/share/mediawiki/includes/mime.types</tt>: application/x-gzip gz tgz # Added tgz (Following the instructions at [http://www.mediawiki.org/ MediaWiki.org] on [http://www.mediawiki.org/wiki/Manual:$wgFileExtensions $wgFileExtensions] and [http://www.mediawiki.org/wiki/Manual:Mime_type_detection#MIME_type_validation MIME type validation].) === Adding a logo === * Create your logo as a 135x135 px PNG file. * Upload it to the wiki. * Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like <code><nowiki>http://site.domain.tld/mediawiki/images/7/77/Logo.png</nowiki></code> : note the path name. * Modify the <code>LocalSettings.php</code> file as shown. $wgLogo = "{$wgScriptPath}/images/7/77/Logo.png"; === Enabling equation display === Download and install the [https://www.mediawiki.org/wiki/Extension:Math Math extension], following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the <code>/var/lib/mediawiki/extensions</code> directory, then loading it. * Install the [https://www.mediawiki.org/wiki/Extension:Math/advancedSettings Tex requirements]: root@server:~# cd /var/lib/mediawiki/extensions/Math/ # install the dependencies listed in the README file e.g.: root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended root@server:/var/lib/mediawiki/extensions/Math# make * Add these lines to the end of <code>LocalSettings.php</code> wfLoadExtension( 'Math' ); // Enable the various rendering modes # $wgMathValidModes[] = 'mathml'; $wgMathValidModes[] = 'latex'; // Set the default rendering option # $wgDefaultUserOptions['math'] = 'mathml'; # $wgMathFullRestbaseURL= '<nowiki>https://api.formulasearchengine.com/</nowiki>'; wgDefaultUserOptions['math'] = 'latex'; * While testing, you might want to add this to <code>LocalSettings.php</code> as well: $wgShowExceptionDetails = true; * In the shell, run the update script, for each wiki in the farm: root@server:~# cd /var/lib/mediawiki/maintenance root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php === Generate sitemaps === Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard [https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php script for generating sitemaps]. * Create a directory for the sitemap: root@server:~# mkdir /var/www/domain.tld/sitemap root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap * If you're rewriting paths, don't rewrite the path to the sitemap. Edit <code>/etc/apache2/sites-enabled/domain.tld.conf</code> to include the line RewriteCond %{REQUEST_URI} !^/sitemap/ : then reload the server. root@server:~# systemctl reload apache2.service * Create a cron job as </etc/cron.d/mediawiki-sitemaps</code> to regenerate the sitemap daily. (This needs to run the script as <code>www-data</code> user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.) 07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \ --fspath /var/www/domain.tld/sitemap \ --server <nowiki>https://www.domain.tld</nowiki> \ --urlpath <nowiki>https://www.domain.tld/sitemap</nowiki> \ --conf /var/www/domain.tld/mediawiki/LocalSettings.php * Submit the sitemap to the search engine. Repeat these steps for every wiki in the farm. === Installing 'Help' pages === Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages: * Help:Contents * Help:Navigation * Help:Searching * Help:Tracking_changes * Help:Editing_pages * Help:Starting_a_new_page * Help:Formatting * Help:Links * Help:Categories * Help:Images * Help:Templates * Help:Tables * Help:Variables * Help:Managing_files * Help:Preferences * Help:Skins * Help:Namespaces * Help:Interwiki_linking (don't include) * Help:Special pages * Template:PD_Help_Page * Template:Admin_tip * Template:Prettytable * Template:Hl2 * Template:Hl3 * Template:Thankyou * Image:Example.jpg * Image:Geographylogo.png * Image:Tools.png * Image:M-en-sidebar.png * Image:M-en-pagetabs.png * Image:M-en-userlinks.png * Image:M-en-recentchanges.png * Template:Click * Template:Languages * Help:Range_blocks * Help:Managing_user_rights * Help:Copying * Category:Help * Category:Category You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template. ===Installing a mobile skin=== The Mediawiki extension [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] gives a different presentation for mobile devices. To install, follow the instructions on the [https://www.mediawiki.org/wiki/Extension:MobileFrontend MobileFrontend] page, placing the extension's <code>.tar.gz</code> file in <code>/var/lib/mediawiki/extensions</code> and expanding it there. Then modify <code>LocalSettings.php</code> to include, at the end # Enable mobile front end wfLoadExtension( 'MobileFrontend' ); $wgMFAutodetectMobileView = true; # wfLoadSkin( 'MinervaNeue' ); $wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards). === Link attributes === The [https://www.mediawiki.org/wiki/Extension:Link_Attributes Link Attributes] extension allows some attributes to be added to links, such as the <code>me</code> attribute needed to verify Mastodon links. Installation is simple, just following the instructions. Download and extract the code, then insert the line wfLoadExtension( 'Link_Attributes' ); into <code>LocalSettings.php</code>. === Increasing upload limits === [https://www.mediawiki.org/wiki/Manual:Configuring_file_uploads#Set_maximum_size_for_file_uploads File upload size limits] are controlled by both PHP and Mediawiki. The PHP limits are smaller by default. To increase the PHP limits, create a file <code>.user.ini</code> in the wiki's root directory (the same one as the <code>LocalSettings.php</code> file) that contains the post_max_size = 10M upload_max_filesize = 10M Then restart the php8.1-fpm and Apache services. root@server:~# systemctl restart php8.1-fpm.service root@server:~# systemctl restart apache2.service If you want uploads of more than Mediawiki's default of 100Mb, you'll need to set <code>$wgMaxUploadSize</code> in <code>LocalSettings.php</code>. == Restoring, and upgrading from previous versions == * Take a backup of the wiki database root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2" * If necessary, create a new, empty database and the user to run it: root@server:~# mysql -u root -p mysql> create database wikidb; mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword'; mysql> grant all on wikidb.* to 'wikiuser'@'localhost'; mysql> quit; * Load the database root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p * Copy all the files from the backup into <code>/var/www/site.domain.tld/mediawiki/</code> root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/ * Delete <code>/var/www/site.domain.tld/mediawiki/Test.php</code> * Rename the <code>LocalSettings.php</code> file to <code>LocalSettings.php.old</code> * Move the <code>uploads</code> direcory to its new name <code>images</code> root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images * Make a few changes in <code>LocalSettings.php</code> ini_set( 'memory_limit'. '128M' ); $wgDBadminuser = "root"; $wgDBadminpassword = "RootUserPassword"; wgUploadPath = "$wgScriptPath/images"; $wgUploadDirectory = "$IP/images"; * Update the database root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php * Remove the <code>$wgDBadminuser</code> and <code>$wgDBadminpassword</code> lines from <code>LocalSettings.php</code> * Add the config files to Apaches root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/ root@server:~# a2ensite site.domain.tld root@server:~# service apache2 reload == See also == Here are a few pages that are useful guides or provide background and context. * [http://www.mediawiki.org/wiki/Manual:Running_MediaWiki_on_Ubuntu_GNU/Linux MediaWiki's guide to installaing MW on Ubuntu] [[Category:Server setup]] 1351e2e396bcb05cd0bb6d634de619d131e4f884